- You have an on-premises Microsoft Skype for Business Server 2015 or Lync Server 2013 enterprise pool.
- You have one or more third-party applications using trusted application endpoints that create a large number of conferences.
- An outage or network disconnect occurs for one or more of your Skype for Business Server 2015 or Lync Server 2013 Front End servers.
During an outage or disconnect of one or more Front End servers, the currently active Front End servers may take ownership of certain sets of users, and in the process, require the creation of new secondary replicas for those users.
When many conferences are created over time by a third-party application, the Persisted Service Data (PSD) table may grow to a size at which replication to the new secondary replicas takes an excessive amount of time to complete. This prolonged process causes queues on the server to become full for longer than expected, and it triggers replication failures during conference operations.
This tool locates any entries related to conferences that were created by application endpoints but that have since been deleted. It also purges those entries from the PersistedServiceData table.
Note If you were actively affected by this issue before running this tool, you may notice that currently affected application endpoints continue to fail during conference operations, even after you run the tool. This issue may persist until the secondary replication process has completed.
You can use the Get-CsPoolUpgradeReadinessState PowerShell command to determine whether all the secondary replication processes have completed. When the state is Busy, there are still secondary replication tasks in progress, and therefore some portions of the application endpoints may still be affected. After the state transitions to Ready, all secondary replication tasks should have completed, and all application endpoints should now be able to complete conference operations.
Follow these steps on every affected pool:
- Optional: Run the following SQL query on the RTCLocal SQL instance of your Front End servers to determine the current number of entries in the PersistedServiceData table. The value returned may vary between individual Front End servers in a particular pool.
select count(*) from [rtc].[dbo].[PersistedServiceData] psd with (nolock)
inner join [rtc].[dbo].[Document] d with (nolock) on psd.DocId = d.DocId
- Run the following command:
PurgeAppEndpointUserDataFromPSD.ps1 -PoolName <pool name> -Command select
- Open the output files and visually confirm that it's safe to delete their data.
- Run the following command:
PurgeAppEndpointUserDataFromPSD.ps1 -PoolName <pool name> -Command delete
- Optional: While the tool is running, you can run the SQL query from step 1 to verify that the count is being reduced. After the script has finished running, you can run the SQL query one more time to determine the final number of entries in the PersistedServiceData table on each Front End server. This number should be significantly less than the count seen in step 1.
Ongoing maintenanceThis tool purges all relevant entries that are currently in the PersistedServiceData table, however, as any third-party applications that use conferences continue to create new conferences, the size of the table will continue to grow. Therefore, we recommend that you run the PurgeAppEndpointUserDataFromPSD.ps1 tool periodically to ensure the size of the PersistedServiceData table does not grow to the point where this issue occurs again.
The frequency with which to run this tool depends on the rate at which these third-party applications create conferences. You can use the SQL query from step 1 in the "Resolution" section to monitor the size of the table and to determine how quickly it is growing. This will help you determine an appropriate schedule for your environment.