Troubleshooting console issues with Data Protection Manager 2012

What does this guide do?

Helps you diagnose and resolve performance and crash related issues with the admin console for System Center 2012 Data Protection Manager (DPM 2012 or DPM 2012 R2).

Who is it for?

DPM 2012 admins who help resolve issues with the admin console.

How does it work?

We’ll begin by asking if your DPM is the latest. If your DPM is up-to-date and you’re still facing the issue, we’ll take you through a series of steps to resolve your issue.

Estimated time of completion:

15-30 minutes.

First Steps

There can be many potential causes for crashes or performance issues with the admin console for System Center 2012 Data Protection Manager (DPM 2012). Many times these are symptomatic of issues with the backend DPM database so the first step in troubleshooting these issues should always be to ensure that the latest Update Rollup has been applied for the version of DPM that is running. There are several known performance issues for DPM that get resolved in each of these rollup packages. For the latest version, you can simply do a Bing search or you can check the article titled List of Build Numbers for System Center Data Protection Manager (DPM) in the TechNet Wiki. Install the latest Update Rollup and then test to verify whether the issue still exists.


Did this solve your problem?

First Steps

There can be many potential causes for crashes or performance issues with the admin console for System Center 2012 Data Protection Manager (DPM 2012). Many times these are symptomatic of issues with the backend DPM database so the first step in troubleshooting these issues should always be to ensure that the latest Update Rollup has been applied for the version of DPM that is running. There are several known performance issues for DPM that get resolved in each of these rollup packages. For the latest version, you can simply do a Bing search or you can check the article titled List of Build Numbers for System Center Data Protection Manager (DPM) in the TechNet Wiki. Install the latest Update Rollup and then test to verify whether the issue still exists.


Did this solve your problem?

Congratulations!

Your Data Protection Manager console issue is resolved.

Sorry

It appears that we are unable to resolve your issue by using this guide. For more help resolving this issue please see our TechNet support forum or contact Microsoft Support.

Performance Issue or Console Crash?

If the problem remains even after installing the latest Update Rollup Package for DPM 2012, we first need to narrow down the nature of the problem.

DPM console crashes

When dealing with console crashes, it’s important to understand that the console on the DPM server relies on several services being available, and if any of these stop running or fail then you will likely see an error similar to the following:



If the crash is happening when launching the console, verify that all of the DPM services are running. The services that need to be running are listed in the error message and include the following:

    • DPM
    • DPMRA
    • SQL Agent (for DPM instance)
    • SQL Server (for DPM instance)
    • Virtual Disk Service
    • Volume Shadow Copy Service

If one of these services isn’t running, try starting it and then reopen the DPM console. If there is a problem starting the service, the error message should give a clue as to the cause of the failure.


Performance Issues

This guide focuses on the most common DB related issues, however if general performance issues are observed across the server as a whole then the server may simply be over utilized. A performance monitor trace is a good place to start if you suspect server-wide performance issues. While server performance evaluation is something best left to experts, we have some general instructions for server performance analysis here: http://technet.microsoft.com/en-us/magazine/2008.08.pulse.aspx?pr=blog.

Within the scope of this guide we will look only at performance issues that are limited to the DPM console with the assumption that the rest of the server is responding normally.

First we want to see if the DPM database is getting bloated (larger than it should be) for any reason, as this can slow down queries returning data and also cause other issues such as filling the C: drive on the DPM server with the database data files.

The check the size of DPMDB, right-click on the DB and select Properties:



If the DB is large then we want to identify which table(s) in the DB are responsible. Large can be fairly subjective as certain configurations (e.g. protection of large Sharepoint Farms) will naturally lead to DB growth, however if it is causing issues with disk space or there are performance issues then these steps should be followed anyway.

To find out which tables are large, run the Disk Usage by Top Table report against the DPMDB as shown below. 



Based on the output of this report, which should list the tables in order of size, take a look at the largest tables. Below are some of the tables that are most often found to be large. Select the table that is large in your environment or select ‘Database and table sizes appear normal’ to go on to the next step.

The Database may be in Recovery Mode

If the database is in recovery mode this can cause problems when services attempt to connect to it. The DB can get put into recovery mode due to a DPMSync failure or due to a crash. To check whether this is the case, run the SQL command below against the DPMDB:

select * from tbl_DLS_GlobalSettingwhere PropertyName like 'DbRecovery'

If the PropertyValue returned is 1 then the DB is in recovery mode.

Run the following to take it out of recovery mode:

updated tbl_DLS_GlobalSettingset PropertyValue = '0'where PropertyName like 'DbRecovery'

Once complete, restart the DPM service and try the console again.


Did this solve your problem?

The service starts but then crashes

If starting the service results in the service crashing, check in the Application Event Log for an error indicating which service has crashed. Check for any entries with Error as the level and MSDPM (or any other DPM service) as the source which corresponds to the time of the crash. The General tab for the event should contain information about the service that crashed and some details about the crash as shown in the example below.


In this example we can see that the failing process is MSDPM and that the Problem Details section tells us that it failed with the error code 0x80004015 which maps to:

The class is configured to run as a security id different from the caller

We can therefore start investigating the issue as a user account issue. Because we know it was the MSDPM service that crashed, the next step is to take a look at the corresponding DPM error log. The default location for these DPM error logs is:

C:\Program Files\Micorosft System Center 2012\DPM\DPM\Temp\

The error logs in this folder will be named for the service they log, and the current log file for each service will be named <service>curr.errlog. 

If the service has crashed, the system should also create a .crash file similar to those shown below.


The crash event should be recorded at the very end of the file and show you further details as to the cause.  

While troubleshooting the various services crashes, their causes and resolutions are beyond the scope of this guide, the Event Logs, error logs and .crash files should provide you with enough information to troubleshoot the most common errors via the Microsoft support site (http://support.microsoft.com), the DPM blog (http://blogs.technet.com/dpm) or the DPM support forum (http://social.technet.microsoft.com/Forums/en-us/home?category=dpm).


Did this solve your problem?

Check the service run-as account

If you are having trouble starting one of the DPM related services then it may be due to the service run-as account. The error below is indicative of an issue with the service run-as account.



The only services that might be running with an account other than SYSTEM are the SQL server accounts. Use the table below to verify that the accounts are correct and that they have valid passwords. Note that to change the SQL user accounts it is best to do it via the SQL Server Configuration Manager interface rather than directly via the Services snap-in. 



Did this solve your problem?

The service is unable to connect to the DPM database

If the service is unable to connect to the DPM database it will likely be unable to start and you will see errors similar to the following:



The Problem Details section in the Event Log error should provide additional information regarding the nature of the failure. Typically the DB is offline/not contactable (more likely if on a remote server), or you may have a login failure. If so you will probably see an error in the Event Log similar to one of the following:


or


Some common reasons why we cannot connect to the DB include:

  • Login failure
  • DB/instance offline
  • Network issue preventing connection

The error message should give you a clue as to the cause of failure.

Login Failures

The account that is failing to login should be clear in the error message, but if not you can also check msdpmcurr.errlog in the DPM Temp folder. If this doesn’t make things clear, try the ERRORLOG files which will be in the SQL install location (e.g. C:\Program Files\Microsoft SQL Server\MSSQL11.MSSQLSERVER\MSSQL\Log). Note that the path may vary depending on the exact version of SQL installed or if it was installed to a non-default location.

This error log file should include any failed login audit entries. Resolve these errors by assigning permissions to the account mentioned for the DB referenced. This is normally either the SQL run-as account or the SYSTEM account.

For the SYSTEM account you can add the relevant permissions in SQL Management Studio by going to Security -> Logins and then right-clicking the System account. Ensure that it has the sysadmin role selected as shown below.


For the SQL run-as account, resetting the account via the Configuration Manager should resolve any issues.


Did this solve your problem?

The DB/instance is offline

You probably already checked that the SQL Server service is running at this point, but if not then check that now. Once you’ve verified that it is running, try to connect to the instance from SQL Management Studio (SSMS). Occasionally this can fail if the server is logged in under a different account than the account it was installed under. In this scenario, try running SSMS as Administrator. If you can connect successfully then the DPMDB is online, however if it is offline it will look like the following:



If the DB is offline, right-click the DPMDB, select Tasks and then select Bring online. Once online, test and see if the problem is resolved.


Did this solve your problem?

Network related issues

If you see errors that suggest there is a network related problem, test the connection to the DB from the DPM server by completing the following:

  1. Create a .udl file. The easiest way is to rename a blank .txt file with a .udl extension.
  2. Double click on the UDL file and select the Instance and DB to test from the dropdown.
  3. Click the Test Connection button.


If this fails, check that you can ping the SQL server from the DPM server and verify that name resolution is working properly. Also verify that the IP address returned is correct. Verify that the address is also correct in SQL -> DPM server. Check for any other obvious reasons why traffic might not get through (firewalls, etc.).


Did this solve your problem?

Service timeouts

If the service run-as accounts are properly configured then you may be experiencing an issue with service timeouts. If the service times-out when trying to start you can try applying the following registry key:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control 

DWord: ServicesPipeTimeout

Value: 300000 

If the DWORD does not exist you can create it. The value is the timeout in milliseconds (ms), so 60000 equals 1 minute (60 seconds). You will need to restart the service to implement the changes. You can adjust the value as necessary and the example above sets the timeout to 5 minutes.


Did this solve your problem?

Tbl_RM_SharePointRecoverableObject

This table can become large when large SharePoint farms are being protected (e.g. farms with millions of items), or multiple farms are protected. The following formula will approximately tell you how big the DPMDB can get when protecting a large SharePoint farm:

((Number of items in the Farm in millions) x 3) + ((number of content DBs x Number of SQL servers in farm x 30) / 1024) = size of DPMDB (Gb)

This only takes into account the SharePoint related DB growth so it may be larger than this value overall, and you may need to work out the size per farm and then total all of them together. There is not much we can do to reduce the disk space used by the DB in this scenario, however if performance is impacted we can inspect the database index size and fragmentation for issues.

Checking DB Index size and fragmentation

Large or highly fragmented indexes can cause significant performance issues for SQL queries running against them and take up more disk space than required. To check for fragmentation, run the following query which will return all the indexes with >20% fragmentation:

SELECT OBJECT_NAME(i.OBJECT_ID) AS TableName,i.name AS IndexName,indexstats.avg_fragmentation_in_percentFROM sys.dm_db_index_physical_stats(DB_ID(), NULL, NULL, NULL, 'DETAILED') indexstatsINNER JOIN sys.indexes i ON i.OBJECT_ID = indexstats.OBJECT_IDAND i.index_id = indexstats.index_idWHERE indexstats.avg_fragmentation_in_percent > 20order by TableName

If there is a large index for one or more table, this will often be resolved by rebuilding the index, particularly if it is very fragmented.

You can run the following query to rebuild an index on a particular table:

ALTER INDEX ALL ON <table_name>rebuildgo

If there are a lot of very fragmented tables, you can run the following to rebuild every index on a DB:

DECLARE @TableName VARCHAR(255)DECLARE @sql NVARCHAR(500)DECLARE @fillfactor INTSET @fillfactor = 80DECLARE TableCursor CURSOR FORSELECT OBJECT_SCHEMA_NAME([object_id])+'.'+name AS TableNameFROM sys.tablesOPEN TableCursorFETCH NEXT FROM TableCursor INTO @TableNameWHILE @@FETCH_STATUS = 0BEGINSET @sql = 'ALTER INDEX ALL ON ' + @TableName + ' rebuild'EXEC (@sql)FETCH NEXT FROM TableCursor INTO @TableNameENDCLOSE TableCursorDEALLOCATE TableCursorGO

This may take some time to run but should not affect the ability of DPM to run as usual, although possibly a little slower during the rebuild.

There is also additional information on rebuilding indexes in the DPMDB at the following link:

http://technet.microsoft.com/en-us/library/hh758188.aspx

If you are seeing issues with SharePoint Catalog creation taking a long time it is recommended to rebuild the indexes for tbl_RM_SharePointRecoverableObject and tbl_RM_RecoverySource.


Did this solve your problem?

TempDB Bloating

This is not as common as bloating of the DPM DB, but it can cause some performance related issues.

NOTE If you want to know more about the TempDB there is further detail at http://msdn.microsoft.com/en-us/library/ms190768.aspx.

Why might it grow?

It could be simply that we have a query working with a large amount of data. This is not likely to be the case if it is growing by several GB. More commonly though, it can be caused by a long running transaction. This can prevent cleanup of the transaction log which causes it to continue growing until the transaction completes, or indefinitely if it never completes.

Also, if other databases are co-located on the same instance it may not be DPM at fault as the TempDB is available to all DBs on an instance. If another application is causing a problem with the TempDB it could therefore impact on DPMs console performance.

To verify whether this is a problem, go to SQL Management Studio and open the TempDB properties:


If it is GB in size then there is likely a problem.

NOTE With TempDB it is not possible to enable AutoShrink, and if AutoGrow is enabled, once the DB grows to a large size it will not drop back in size. For more information on AutoShrink and AutoGrow please see the following:

315512 - Considerations for the "autogrow" and "autoshrink" settings in SQL Server (http://support.microsoft.com/kb/315512) 

In Properties you can also see if there is any currently free space (Space Available) in the TempDB. If so, it is unlikely that whatever caused the growth is currently happening.


There is available free space in the TempDB

From the SQL management console you can try to shrink the TempDB size:


If there is available free space in the DB you should be able to shrink it. If you find that the size does not decrease as much as expected, check what the initial size of the DB is set to:


Sometimes this value can be set quite large. If so, drop it back to 8Mb and try the shrink again. Restarting the SQL instance will also reset the TempDB size back to its initial size.


Did this solve your problem?

There is no free space in the TempDB and it is currently large and/or growing

In this scenario, it is likely the problematic query is still running. If it is causing an urgent issue (e.g. it is about to consume all available disk space) restarting the SQL instance should bring temporary relief. Keep in mind however that this will make it harder to identify the source of the problem.

If the issue is due to a long running query, the script below will show you the longest running transactions:

SELECT transaction_id, session_id, elapsed_time_secondsFROM sys.dm_tran_active_snapshot_database_transactions ORDER BY elapsed_time_seconds DESC;

That will return something similar to the following:

Used in conjunction with a SQL profiler trace, you should then be able to work out what is happening in the session by filtering on that session_id:


From this you can see which DB the query is running against. The details pane at the bottom should then show you the text of that query. First check that the DatabaseName is DPM DB. If not then it is another application that is causing the issue and you will need to troubleshoot that application.If it is the DPMDB, take a look at what the query is doing. 

If it is calling a stored procedure you can take a closer look at the stored procedure by opening SQL Management Studio and locating the procedure under DPMDB -> Programmability -> Stored Procedures:

IMPORTANT Before making any changes, be sure that you have a backup of the DPMDB! Right-click the appropriate stored procedure to see more details. Clicking modify will allow you to see the actual query. You can then take a look at any of the tables we are selecting from (or updating) to check if they have any problems. While it is beyond the scope of this troubleshooter to resolve complex SQL table and query issues, this should help pinpoint where the problem may reside. Consult your local SQL expert for more information. You can also search for more information using the Microsoft support site (http://support.microsoft.com), the DPM blog (http://blogs.technet.com/dpm) or the DPM support forum (http://social.technet.microsoft.com/Forums/en-us/home?category=dpm). 


Did this solve your problem?

Tbl_ARM_DirAndFile and/or Tbl_ARM_Path

The tape catalog pruning settings can cause these tables to be quite large. To reduce the size of these tables, modify the tape catalog retention values to reduce the amount of data being stored in DPMDB:


The default is to remove the entries as the tapes expire. If tapes are kept for multiple years however, this can lead to a large amount of data retained in the DB. Change the settings to “Prune catalog for tapes older than” and set it to a lower value such as 4 weeks as shown below.


Updating this will NOT delete data on tape, but it will mean that tapes older than this value will need to be recataloged in order to restore data from them. Make sure that you are complying with any data retention policies for your organization before making any changes.


Did this solve your problem?

Tbl_TE_TaskTrail

If this table is growing large then it is generally a sign that the overnight jobs to clean up the DB are failing, as Garbage Collection should clean up any data older than 33 days in the task trail table.

To check if Garbage Collection is doing its job, run the following query:

select * from tbl_TE_TaskTrail ttwhere datediff (day, tt.stoppeddatetime, getutcdate()) > 33

This will tell you how many entries there are that are greater than 33 days old. If 0 then Garbage Collection is working properly. You can manually clean up the tables using the script below which will normally clear the problem.

IMPORTANT Be sure that you have a current backup of the database before running any script!

------------------------- start --------------------------USE [DPMDB_MUKESH]GOselect * into dbo.tbl_TE_TaskTrail1 from tbl_TE_TaskTrail where IsGCed = '0'IF EXISTS (SELECT * FROM sys.foreign_keys WHERE object_id = OBJECT_ID(N'[dbo].[fk_TE_TaskTrail__JM_JobTrail]') AND parent_object_id = OBJECT_ID(N'[dbo].[tbl_TE_TaskTrail]'))ALTER TABLE [dbo].[tbl_TE_TaskTrail] DROP CONSTRAINT [fk_TE_TaskTrail__JM_JobTrail]GOIF EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID(N'[DF_tbl_TE_TaskTrail_IsGCed]') AND type = 'D')BEGINALTER TABLE [dbo].[tbl_TE_TaskTrail] DROP CONSTRAINT [DF_tbl_TE_TaskTrail_IsGCed]ENDGOUSE [DPMDB_MUKESH]GOalter table dbo.tbl_AM_AgentDeploymentTrail DROP CONSTRAINT fk__AM_AgentDeploymentTrail__TE_TaskTrail__TaskIDalter table dbo.tbl_ARM_TaskTrail DROP CONSTRAINT fk__ARM_TaskTrail__TE_TaskTrail__TaskIdalter table dbo.tbl_CM_InquiryResult DROP CONSTRAINT fk__CM_InquiryResult__TE_Task__TaskIDalter table dbo.tbl_MM_MediaRequiredAlert DROP CONSTRAINT fk__MM_MediaRequiredAlert__TE_TaskTrailalter table dbo.tbl_MM_Task DROP CONSTRAINT FK_tbl_MM_Task_tbl_TE_TaskTrailalter table dbo.tbl_MM_TaskTrail DROP CONSTRAINT fk__MM_TaskTrail__TE_TaskTrail__TaskIdalter table dbo.tbl_PRM_CloudRecoveryPointTrail DROP CONSTRAINT fk_PRM_CloudRecoveryPointTrailalter table dbo.tbl_PRM_ReferencedTaskTrail DROP CONSTRAINT fk__PRM_ReferentialTask_TaskTrail_TaskIdalter table dbo.tbl_RM_CandidateDatasetsForSCAssociation DROP CONSTRAINT fk_RM_CandidateDatasetsForSCAssociation_RM_TaskTrailalter table dbo.tbl_RM_RecoveryTrail DROP CONSTRAINT fk__RM_RecoveryTrail__TE_TaskTrail__TaskIdalter table dbo.tbl_RM_ReplicaTrail DROP CONSTRAINT fk_RM_ReplicaTrail__TE_Taskalter table dbo.tbl_RM_ShadowCopyTrail DROP CONSTRAINT fk_RM_ShadowCopyTrail__TE_Taskalter table dbo.tbl_TE_TaskError DROP CONSTRAINT FK_tbl_TE_TaskError_tbl_TE_TaskTrail/****** Object: Table [dbo].[tbl_TE_TaskTrail] Script Date: 07/18/2011 00:25:30 ******/IF EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[tbl_TE_TaskTrail]') AND type in (N'U'))DROP TABLE [dbo].[tbl_TE_TaskTrail]GOUSE [DPMDB_MUKESH]GO/****** Object: Table [dbo].[tbl_TE_TaskTrail] Script Date: 07/18/2011 00:25:30 ******/SET ANSI_NULLS ONGOSET QUOTED_IDENTIFIER ONGOCREATE TABLE [dbo].[tbl_TE_TaskTrail]([TaskID] [dbo].[GUID] NOT NULL,[TaskDefinitionID] [dbo].[GUID] NOT NULL,[JobID] [dbo].[GUID] NOT NULL,[VerbID] [dbo].[GUID] NOT NULL,[ExecutionState] [smallint] NOT NULL,[ErrorCode] [int] NOT NULL,[ErrorDetailsXml] [ntext] NOT NULL,[LastStateName] [nvarchar](128) NULL,[CreatedDateTime] [datetime] NOT NULL,[StartedDateTime] [datetime] NULL,[StoppedDateTime] [datetime] NULL,[TaskXml] [ntext] NOT NULL,[ErrorStateName] [nvarchar](128) NULL,[IsGCed] [bit] NOT NULL,CONSTRAINT [pk_TE_TaskTrail] PRIMARY KEY NONCLUSTERED ([TaskID] ASC)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY]GOinsert into tbl_TE_TaskTrail select * from tbl_te_tasktrail1ALTER TABLE [dbo].[tbl_TE_TaskTrail] WITH CHECK ADD CONSTRAINT [fk_TE_TaskTrail__JM_JobTrail] FOREIGN KEY([JobID])REFERENCES [dbo].[tbl_JM_JobTrail] ([JobId])GOALTER TABLE [dbo].[tbl_TE_TaskTrail] CHECK CONSTRAINT [fk_TE_TaskTrail__JM_JobTrail]GOALTER TABLE [dbo].[tbl_TE_TaskTrail] ADD CONSTRAINT [DF_tbl_TE_TaskTrail_IsGCed] DEFAULT ((0)) FOR [IsGCed]GOalter table dbo.tbl_AM_AgentDeploymentTrail ADD CONSTRAINT fk__AM_AgentDeploymentTrail__TE_TaskTrail__TaskID foreign key (taskid) references tbl_te_tasktrail (taskid)alter table dbo.tbl_ARM_TaskTrail ADD CONSTRAINT fk__ARM_TaskTrail__TE_TaskTrail__TaskId foreign key (taskid) references tbl_te_tasktrail (taskid)alter table dbo.tbl_CM_InquiryResult ADD CONSTRAINT fk__CM_InquiryResult__TE_Task__TaskID foreign key (taskid) references tbl_te_tasktrail (taskid)alter table dbo.tbl_MM_MediaRequiredAlert ADD CONSTRAINT fk__MM_MediaRequiredAlert__TE_TaskTrail foreign key (taskid) references tbl_te_tasktrail (taskid)alter table dbo.tbl_MM_Task ADD CONSTRAINT FK_tbl_MM_Task_tbl_TE_TaskTrail foreign key (taskid) references tbl_te_tasktrail (taskid)alter table dbo.tbl_MM_TaskTrail ADD CONSTRAINT fk__MM_TaskTrail__TE_TaskTrail__TaskId foreign key (taskid) references tbl_te_tasktrail (taskid)alter table dbo.tbl_PRM_CloudRecoveryPointTrail ADD CONSTRAINT fk_PRM_CloudRecoveryPointTrail foreign key (taskid) references tbl_te_tasktrail (taskid)alter table dbo.tbl_PRM_ReferencedTaskTrail ADD CONSTRAINT fk__PRM_ReferentialTask_TaskTrail_TaskId foreign key (taskid) references tbl_te_tasktrail (taskid)alter table dbo.tbl_RM_CandidateDatasetsForSCAssociation ADD CONSTRAINT fk_RM_CandidateDatasetsForSCAssociation_RM_TaskTrail foreign key (taskid) references tbl_te_tasktrail (taskid)alter table dbo.tbl_RM_RecoveryTrail ADD CONSTRAINT fk__RM_RecoveryTrail__TE_TaskTrail__TaskId foreign key (taskid) references tbl_te_tasktrail (taskid)alter table dbo.tbl_RM_ReplicaTrail ADD CONSTRAINT fk_RM_ReplicaTrail__TE_Task foreign key (taskid) references tbl_te_tasktrail (taskid)alter table dbo.tbl_RM_ShadowCopyTrail ADD CONSTRAINT fk_RM_ShadowCopyTrail__TE_Task foreign key (taskid) references tbl_te_tasktrail (taskid)alter table dbo.tbl_TE_TaskError ADD CONSTRAINT FK_tbl_TE_TaskError_tbl_TE_TaskTrail foreign key (taskid) references tbl_te_tasktrail (taskid)DROP TABLE [dbo].[tbl_TE_TaskTrail1]GO


Did this solve your problem?

गुण

आलेख ID: 10057 - पिछली समीक्षा: 08/03/2016 - संशोधन: 103

प्रतिक्रिया