Provide Feedback on this Broadcast

Note This document is based on the original spoken Support WebCast transcript. It has been edited for clarity.

Irfan Soomro: My name is Irfan, and I'll be talking about the Recovery Storage Groups. The objectives for this presentation (slide 2) are to discuss the new Recovery Storage Group feature, which has been introduced in Microsoft® Exchange 2003, how the Recovery Storage Groups can be used in disaster recovery scenarios, and what other methods are available for mailbox recovery. The other methods available for mailbox recovery will be just a brief talk; it won't go in depth.

So some of the specific topics that I'll talk about in detail (slide 3) include how Recovery Storage Group features work, how to set up a Recovery Storage Group, how to restore mailbox data to Recovery Storage Groups, and how we can recover the data from them using the Recovery Storage Group. I'll try to give a live demo of all the slides or screenshots that I'll be showing during the presentation, and hopefully we'll start with adding a Recovery Storage Group all the way through the process of doing an Exmerge out of it.

So getting started (slide 4), what is a Recovery Storage Group? Why do we need the Recovery Storage Group, first of all? The need for this kind of concept comes from the fact that in Exchange 2000 the disaster recovery or mailbox recovery was a very lengthy task. You had to mount another copy of the database or different versions of production databases, and set that Active Directory® forest and do a recovery server. This specifically requires that a spare server be available all the time, tying up your resources, and also requiring time, about two or three hours or maybe a little more, to set up this environment. This prompted us to go into a direction to introduce the Recovery Storage Group, which can be used on the live server. You don't need, in most cases, a recovery server to be present.

How does this Recovery Storage Group work? You can consider Recovery Storage Groups to be special or additional instances of a normal storage group in the Exchange Server that is used exclusively for restoring the databases for mailbox or database recovery. There are specific circumstances where we can use it, and there are some scenarios where we cannot use it.

So how does this special Recovery Storage Group differ from the other storage groups? In most respects we could say that Recovery Storage Groups (or RSGs as I'll be referring to them during the presentation) are similar to an ordinary storage group, but several features and functions that are offered in an ordinary storage group have been disabled on the Recovery Storage Group. This is really done to minimize the performance overhead of having a Recovery Storage Group on the same server, and to prevent the problematic interaction between Recovery Storage Groups and the rest of the Exchange organization which is in production, and also to minimize the likelihood of errors in configuring the Recovery Storage Group that could take the wrong data into the wrong area.

So Recovery Storage Groups, what are the specific differences? One is the disabled protocols. In the Recovery Storage Group, all mail protocols, except MAPI, are disabled, like SMTP, POP3, IMAP4, etc. They're all unavailable for all the databases within the Recovery Storage Group. This will mean that mail cannot be sent to or from a database while it is in the RSG, and also to prevent adding or removing mail from these databases.

Also there is mailbox connectivity with your access restrictions. The user mailboxes in the Recovery Storage Group cannot be used like ordinary production mailboxes, in the sense that they cannot be connected to Active Directory® accounts. One cannot login to mailboxes like a normal method and connect to that storage area within the Recovery Storage Group. The only supported interface to access these mailboxes and their contents is the Exchange 2003 version of Exmerge; the previous ones are a little different.

In these policies, the system and mailbox management policies are not supported within Recovery Storage Groups. One could say the benefits here would be the prevention of Recovery Storage Groups being deleted by the system when we are working on them, depending on where the policies are.

Maintenance and other tasks are also disabled, like online maintenance and defragmentation. They do not run against the other databases within the Recovery Storage Group. The reason here would be that these databases are not really meant for production and they're only there temporarily for the specific data recovery purpose. Running maintenance or defragmentation is not really needed. The main intention here is not to optimize the database because it is a temporary recovery situation.

So what are the uses for Recovery Storage Group (slide 5)? You can recover mail items for particular users. You can mount blank mailbox stores because of corrupted stores like dial-tone strategy. We'll be covering both of these in a little detail as we go along. I'll also talk about where we cannot use the Recovery Storage Groups, as I mentioned earlier.

What are the situations where we can use it? As I mentioned, recovery is limited to a single mailbox, a single database, or a group of databases within a single storage group. In these situations we can use RSG, and when the logical information in Active Directory about these storage groups and mailboxes is still intact and unchanged.

Some typical scenarios would be recovering deleted items that have been purged from a user's mailbox, recover or repair an alternate copy of a database while the one in production continues to be in use. So the same database that is being used in production that people are using, but you cannot do some kinds of production. You cannot back it up, so you would need to restore a previous copy and swap it with a production database. We can do this using Recovery Storage Group without disruption of service, other than restoring the database for swapping the files, which will be a short duration. Then subsequently, we can either swap the database as I mentioned or we can use Exmerge to merge data between the two databases. I will talk about that later.

A very important thing situation where we cannot take advantage of the Recovery Storage Groups is if you need to restore the entire server. In that case, we need to restore the databases from multiple storage groups, or if the need is to change or enable Active Directory topology or do a complete restore recovery, then you'll need to recover the server and you'll not be able to use Recovery Storage Group. Also from the recovery are public folder stores, then you cannot use RSG. RSG is only meant for mailbox data recovery. If you need to recover public folders, the method is still the same as we have been using in Exchange 2000, so no change there.

So let's see what happens when you bring the databases into the Recovery Storage Group (slide 6), how do they link back, or how do they understand what needs to be done, how do they link back to the original database that is in production. In most of the cases we mentioned earlier, the databases are linked back to the original database; otherwise, we cannot use it.

This linking or ratification really involves two Active Directory attributes that were used to identify the database to be valid within Recovery Storage Group. One is the msExchMailboxGUID attribute. This is a user object property that owns the mailbox. The other one is the msExchOrigMDB attribute. This is a database object property in the Recovery Storage Group. So a mailbox property and a database property both need to be validated.

So how does it work, really? A mailbox must first be checked to be a valid candidate for the data recovery in the Recovery Storage Group. This is decided by its GUID, and must be found on the user object in Active Directory. This is important because each mailbox in the Recovery Storage Group is in a disconnected state, and I will show you what I mean by that in a screenshot. Exmerge cannot access a mailbox that is disconnected under normal circumstances. So by linking the mailbox in the Recovery Storage Group back to the original database and the original owner, Exmerge can still log on to the database using the original mailbox logon credentials and recover the data from it.

The downside is that the deleted mailbox cannot be recovered by a Recovery Storage Group, because deleting a mailbox will remove the mailbox attribute from the user object that owned the mailbox, and hence, Exmerge cannot identify it. So the mailbox GUID is the only unique and distinct attribute of the mailbox, and it's the same for the lifetime of the mailbox. If we modify it in any way, a character here or there, it is essentially a new mailbox as far as the infrastructure of Exchange is concerned.

The msExchOrigMDB is the second condition that we have to pass, in addition to the mailbox GUID. The trick here is that the database must belong to the original database whose backup is being used in the Recovery Storage Group. From the time of the backup and the restore, if the mailbox has been moved to a different database, then you will still see the mailbox listed in Exmerge because it has the GUID in Active Directory, so it's a valid mailbox. But we will not be able to extract data from it because the mailbox itself does not belong to the database that is in that Recovery Storage Group at the moment.

If you need to move the mailbox to another storage group, then you have two options to temporarily override the behavior. One is to either move the mailbox back to the original database before running the Exmerge, or you can go in and modify the msExchOrigMDB attribute in the Active Directory on the Recovery Storage Group database, and point it to the database where the mailbox now resides. But the downside of this method is that it will restrict the recovery of only that particular mailbox, which has been moved, because we are modifying the attribute for that and considering that database — one mailbox is pointing to the new location, whereas all the others belong to a different msExchOrigMDB attribute. So depending on the situation, we can follow one or the other.

Let's see Recovery Storage Group override (slide 7). This is an important concept that I wanted to dedicate a slide to. I'll explain why it is necessary and how we do it. The important point here is that, when we add Recovery Storage Groups within an Exchange 2003 environment, it alters the restore behavior from the previous versions of Exchange. It is important to look a little deeper at how backup and restore works. In fact, I would show only the restore part because the backup has not changed, will not change, but restore on a server where there is a Recovery Storage Group will change.

In the sense that, when you have the Recovery Storage Group present and you do a restore, the Exchange information is stored automatically and will redirect all the restores to the Recovery Storage Group. This redirection is completely transparent to the backup programs. If administrators are not aware of this phenomenon, they may see results that they were not expecting, at least from the perspective of the previous version of Exchange. We find that we do a restore, but it is not going to the production area of the storage groups.

Sometimes after a process, when you are using an Exchange-aware backup program to restore the database, it first checks whether a Recovery Storage Group exists on the designated Exchange server. If one does exist, then a second check is done to see if the database that is being restored is still in the Recovery Storage Group; that is, it has been added manually to the RSG. If it has, then the restore will invariably be redirected to the Recovery Storage Group.

The backup program remains unaware that restore has been redirected, because as I mentioned earlier, the redirection is completely transparent to the backup program. However, because the database is not configured in the Recovery Storage Group's resource, the process will terminate, despite that fact that the database is still present in another storage group on the server. This occurs because after you have run the Recovery Storage Group, it is really the only task or recipient of all the restores. Or you could say that as long as the Recovery Storage Group exists, it is treated as if it were the only storage group on the server for restore purposes.

Deleting the restore's RSG, we can immediately divert information stored in normal restore behavior. It does not require you to restart information to restored servers, but in some cases you may have to restart the backup program for the change to take effect. That depends on the program you're using.

Once the recovery is complete and you are done with the work, then the recommendation is that it's good to remove all the databases from the RSG, and then delete the RSG group itself so that normal restore behavior is available in the production environment on an ongoing basis. If you use the RSG in place, the regular server-level restore will also route all the data to RSG. This is why we need the Recovery SG Override setting in the registry.

So what happens if RSG is present on the server, and you attempt to restore a store that is not part of the Recovery Storage Group, or a mailbox store with msExchOrigMDB attribute I mentioned earlier? We'll get a specific event ID 9634 in this case, which will say, "Failed to find a database in Recovery Storage Group," with the original MDB at the end, and I think the error number that you will see in this case is 8004010F.

But there may be a situation there we do not want to delete the Recovery Storage Group; we want to leave it there. You do not want to remove the storage group and the databases. You can do an interim workaround here. This registry key that I mention on slide 7 can be put in place. The data value is set to 1 for the restore purpose, the restore program will see that there is no Recovery Storage Group present, and will then revert to the normal restore behavior. Instead of deleting and re-creating the Recovery Storage Groups, you can have this registry key handy and then modify it accordingly.

Then from this slide (8) on I will show some screenshots for how to help create a Recovery Storage Group and add a mailbox store to it. I will also try to give a live demo of these slides in the end if time permits.

Once you are Exchange system managers (slide 9), you right-click the server name and select New. In this case, you will see you get two options: one is Storage Group and the other is Recovery Storage Group. After you select Recovery Storage Group, you get another interface giving you the transaction log locations, system path, and all that. We'll make a note here that the log file prefix shown here is R00 rather than E00. The differences are made so that we can segregate or differentiate between the log files and checkpoint of databases within the normal production, and the ones that are in the Recovery Storage Group.

After we have created the Recovery Storage Group (slide 10), the RSG itself will show up within the Server list of all the storage groups. In order to use it, we have to add databases to the Recovery Storage Group section. Here, creating or adding a database within RSG is different than creating a database in the ordinary storage groups, in the sense that only a list of existing databases to pick from is shown. You are not given a choice or an option to define new databases. In the first instance, if you have multiple storage groups, you will see databases from all those storage groups. As you can see here, there are databases from more than one storage group.

For a database to appear on the RSG pick list, it should meet certain characteristics. The database must currently exist in Active Directory; that is, you can find it in the Exchange System Administrator. (1) The database itself is from a server that is a member of the same administrative group as the Recovery Storage Group server. Or (2) the database is from a server that is running a version of Exchange between Exchange 2000 SP3 and Exchange Server 2003 on the RSG server. Also, if any database already has been added in the RSG, the additional databases that can be added or picked up from must belong to the same storage group as the first database. We will see that in the next slide. All these databases, of course, as I mentioned earlier, have to be mailbox stores. They cannot be public folder databases.

We saw that in the first instance of starting with, we see the databases from all the storage groups (slide 11). After we have selected a particular database and we want to add another one, we will only see the database from that storage group, and all others are not being shown here on the left-hand screenshot. After you have added all the databases and you try to add another database, you will get this error, c1038bf6, which is really self-explanatory — it says that all databases which were available in the original backed up storage group have been added to the recovery storage group, and none is available to add here.

When we are adding a mailbox store to Recovery Storage Group (slide 12), there are some important considerations. These are the database name, the "database can be overwritten" setting on the database properties, and the file locations for the databases.

You can use any name for the database in the Recovery Storage Group. It is not really important here. It will then be stored correctly in the RSG, despite the difference in the name. This applies to the logical database names, not the database file names themselves. This behavior is different from the restore behavior in ordinary storage groups, where one can restore an online backup. Only the name of the database in the backup matches the logical name of the database on the restore server. So there's a little difference in the naming conventions, a little flexibility here, again, to help differentiate between. The reason, really, is that this database in RSG has this msExchOrigMDB attribute that links the database to the original database. So that's why we can be a little flexible.

File locations or database overwrite settings. When you add a database to the Recovery Storage Group, you should leave the This database can be overwritten by a restore check box selected. The important thing to remember about this is, if you restore the same database a second time to the Recovery Storage Group, you will have to mount the check box again before you can mount a second restore of the database. If you don't, Exchange will prompt you about what to do. It will display a specific message, which goes "The database files in this store will be replaced with an older version by an offline restore" and basically it will ask you to select The database can be returned by restore message and you will see 104173, and then the file locations for the database itself.

Another consideration here is disk space. If you don't have sufficient disk space, then you will have an increased downtime when recovering from a disaster. If you restore or copy the database to the same drive as the original database, it may affect the performance a little. The consideration is what type of recovery you are looking for. If the recovery is performed to a copy that is in the Recovery Storage Group database and back into the original storage group, back into Data Swap or Dial Tone, then it is better to have this database on the same logical drive as the original one because it helps in a faster move from one location to another.

Naming conventions we have already discussed. The naming convention is a little flexible here. But if the intent is to move a Recovery Storage Group database back into its original storage group, then the file names must match. This means that they must either use the same file names when creating the Recovery Storage Group or manually rename them before the actual move of the database files.

Salvaging or recovering data from the Recovery Storage Group (slide 13). I've talked about the design goals and features and how you create the Recovery Storage Group. Let's see what the actual process of data recovery is (slide 14). The mailbox stores are restored in two ways. One is online backup, bringing back an online backup, or bringing an offline copy into the Recovery Storage Group. I'll be focusing primarily on the online process in this slide's discussion. There is one little thing to know about the offline copy, and I will mention it later.

Before we start the restore from the backup (slide 15), we should dismount and delete any mailbox stores that are currently mounted in the Recovery Storage Group, and remove logs and checkpoint files in order to ensure a successful recovery. In other words, those directories should be empty before we initiate a restore.

When we are doing the restore of the full backup sets, a condition is that we do not intend to bring any incremental or differential backups, or we're not bringing in additional logs to be played into the database, then we should select the Last Restore Set check box, so that our recovery can be run automatically after the restore is finished. As I mentioned, another point to know is that we should have the Database can be overwritten by restore option already checked in the Database Properties page.

So when we have restored the database into the Recovery Storage Group and the databases are mounted, all the mailboxes will appear as disconnected (slide 16). This is the normal behavior. As I discussed earlier, these mailboxes within the Recovery Storage Group cannot be connected to the accounts. They will remain disconnected. The only way to attach this is with the Exchange 2003 version of Exmerge.

One point that I would like to mention is that, if you are bringing an offline copy back into the Recovery Storage Group, these mailboxes may appear shown as connected. The UI may not show red signs. In reality, they are not connected; they are still disconnected. This is a known issue that I would like to mention so that we can avoid any confusion there. This anomaly is only visual. It is not specific to the interface level, because behind the scenes is still disconnected, and we can continue our normal mailbox recovery with Exmerge. I just wanted to make a note of that.

Shown here (slide 17) is the directory structure of Recovery Storage Group, after the database's other files have been restored to the server. You will see here that we have two sets of files and checkpoint, E00 and R00. As I mentioned, E00 is the original naming convention and R00 is for the segregated files specific to the Recovery Storage Group. The point here is that the logs from E00 would be played first into the database. The database would then be detached and reattached to the R00 series of log files and so on.

Now that we have restored the databases, the system is ready for recovery (slide 18). If the purpose here was to recover the entire database without any consideration for handling any incremental data, then we can just swap the database from Recovery Storage Group to production storage groups. However, if the reasons here are to retrieve one or more mailboxes, then we need this specific tool called Exmerge to connect to the database and to subsequently retrieve data from it, and push it to the production database.

However, I'm sure many of you people are already familiar with Exmerge. We have been using this for quite a long time. But just for the sake of completeness and assuming some people may be new to this, I will briefly go through it again. Make a note that, for the purpose of this presentation, or from the perspective of Recovery Storage Groups, when I refer to Exmerge, it is the Exchange 2003 version of Exmerge. I may not be repeating the whole thing again.

Any previous versions of Exmerge cannot be used against Recovery Storage Group databases. Something else good here is that previously Exmerge was available as part of the resource kit, and during some of the troubleshooting call scenarios it was difficult to have the Exmerge available on the server. If the server goes down, we cannot use e-mail and things like that. So now it is available via download from the Exchange download Web site. The link is mentioned here on slide 18. This method helps us really to ensure that the publicly available tool is always up to date with the latest build. It is available and easy to download the latest build all the time.

So what is Exmerge? We'll go briefly into it. Exmerge is really a two-step process of attaching to a database and, in step one, extracting all the data from mailboxes and the second step is uploading or pushing the data back to another database. This process can also be done by a single step, but in recovery scenarios where you are using Recovery Storage Groups, you will be going through the two-step process.

The intermediary file is a .pst file. Step 1 creates .pst files, and we can use the same .pst files in step 2, to merge the data back to the target stores. All the capabilities of Exmerge that are available for using in ordinary storage groups are also available for databases which are running in the Recovery Storage Group, including the one-step merging, as I mentioned, or the filtering of messages and folders, extraction of rules and permissions.

There are a few differences that I should mention here. When we extract mailbox data to a .pst file, we do not have to override the administrative deny, if you have permissions to extract from the Recovery Storage Group. But when we are merging the data back to the targeted store, in that case we need to change or modify the permissions so that the administrator has the proper permissions on the destination mailboxes. By default, Exchange usually denies these administrative accounts unless you have permissions on the mailbox. When you are using this Exmerge 2003 version, do the special administrative logon that does not require you to grant administrative and override to deny permissions.

The original mailbox here, however, must still be present in the original database, and must still be connected to an Active Directory account or Exmerge to function on that. If the mailbox in the original database on the production database has been disconnected, Exmerge will not show up and show that mailbox on the list of available mailboxes. If all of the mailboxes have been moved to a different database, then it will show up but will not extract data, as I had mentioned a few minutes earlier.

If the mailbox is disconnected and then reconnected to a different Active Directory user, but is still present in the original database, Exmerge can still extract data from the Recovery Storage Group. However, in this case the display name of the mailbox you will see in Exmerge will match the mailNickname attribute of the current mailbox owner, not the previous one, and the directory name seen in Exmerge will match the previous owner's real nickname. The .pst files that are extracted by Exmerge will be based on the old mail nickname, and they will have to be renamed to match the new mail nickname, in this case, before Exmerge can push the data back to the targeted store.

These are some of the details about Exmerge and some limitations and considerations (slide 19). When you are using Exmerge and you point it to the server containing Recovery Storage Group, you will see the databases option available in the Exmerge pane. What is new is if you try to use Exmerge previous to 2003, you will not be able to attach to these Recovery Storage Group databases. I just wanted to show this screenshot, so that you would be familiar with how it would look.

Let's see a couple of other methods available for restoring mailboxes in Exchange 2003 (slide 20). We'll talk briefly, as I mentioned earlier. One is using Exchange System Manager. This method is used if the retention settings for deleted items are configured for mailboxes, and the time that deleted mailboxes can remain on the server has not been expired. This is really a mailbox reconnect scenario.

The other method is to restore a mailbox from a backup to a recovery server (slide 21). As I mentioned, there could be times when you need a recovery server as well. You can use this method if the mailbox object and the Recovery Storage Group cannot be matched to a user object in Active Directory that has the same msExchMailboxGUID value.

This could happen either if the user account is deleted from Active Directory or if the Exchange attributes have been removed from the user account, and the mailbox itself (the Exchange mailbox store) was not reconnected or was deleted. In this specific scenario we will need a recovery server that has sufficient storage capacity to install Exchange and restore all the databases.

Another strategy to use for Recovery Storage Group is complete database recovery with a dial tone database (slide 22). If we have a large mailbox store and we need to restore it, it may take several hours to restore the mailboxes from backup. Using this dial tone strategy or using dial tone database, you can restore the e-mail service to the users quickly with a temporary database or a dial tone database. We can do that restore in parallel. When the database has been restored and is available, we can quickly start the database and do Exmerge to combine the differential data.

So first we reset the database by removing the current database and creating a temporary dial tone. Users can start using this when they have e-mail connectivity. We are doing the restore in parallel and then swapping. These are essentially the steps. This happens because when you create the new dial tone database, the new mailboxes retain the same values for msExchMailboxGUID attribute in the dial tone database itself because it's coming from Active Directory as in the original database. Because this attribute is present, we can use Exmerge to take the small amount of differential data from the dial tone and put it back into the restored database.

When we have the dial tone database running and users have their e-mail functionality, and after we have restored the database Recovery Storage Group, we can also do the repair on these. When the repair operation or restore or however we want to approach it has been completed, we dismount both the databases, swap the database file between the original storage group and the Recovery Storage Group, and mount the databases.

After we do that the users will have access to their previous data, but they will not be able to access the new data that came in when the dial tone or temporary database was in production. In this case, we will need Exmerge to extract that differential data from this one database and put it back on the production one, so that the current production database is complete with all the data. This method is very useful if the new store has some kind of corruption, like you are seeing 1018 errors and are trying to restore an old backup in the Recovery Storage Group.

There are also good reasons for swapping the databases, rather than using Exmerge to move the data out of the old database and into a new one. It really depends on the situation and what we are trying to do. There are certain considerations that we should have in mind when doing the Exmerge as to which way Exmerge should run from the old to the new database or from the dial tone database to the old store.

With the dial tone method, you can manage the database size because the single instance storage will be preserved if you are swapping the databases. The older mailboxes and other such things will be preserved in the state that they were in before the disaster happened, and users will not have to modify their rules or custom folders or things.

As for the time required to complete the merge, if we are bringing the larger database into production and merging from the smaller database, it will require much less time than the other way around. Typically, the dial tone database will have only a few hours worth of mail that it received during the time that you were restoring or repairing the database.

To sum the options up, there are two different strategies that can be used: move database files between the folders in the same drive, or move database files between different drives, or leave the databases in place and then swap the logical parts for the database; these are the different ways we can do the swap.

Recovery Storage Group can also be applied to Exchange Server 2000 with SP3 or later, but it has to be in the same administrative group (slide 23). Other versions of Exchange will not be available in the list of stores that you will see when you are trying to add databases to the Recovery Storage Group.

One thing to note here is that if we use Exchange 2003 Recovery Storage Groups to restore Exchange 2000 SP3 or later databases, the database itself will be upgraded to the Exchange 2003 format after the restore is done. So those databases cannot be used on Exchange 2000 anymore because of the difference in the format. If we try to attempt this, the System Manager displays a warning on the screen. So if you are in that situation, you'll know exactly what went on.

So what are some common errors or problems that are encountered with Recovery Storage Groups (slide 24)? "The specified computer is not a Microsoft Exchange server." This is a fairly generic error, and could occur for several reasons. The store service on the target Exchange server is not running. The name of the Recovery Storage Group is not the same as the name of the original storage group. You are trying to restore a database from a different server or Recovery Storage Group. The Database can be overwritten by restore option has not been checked in Exchange System Manager on the properties of the database object.

Then there's the "database was not found on the target server specified for restoration" error. This is normally caused by Active Directory application latency. If this happens, just wait for a few minutes and try again. Hopefully, this error will not occur in subsequent attempts. Also, if you have set the Recovery Storage Group overwrite key, then you could also come across this error.

Then there's another "database not found" error, C7FE1F42. This will happen if we have the Recovery Storage Group overwrite key in place or in enabled mode. Another one is, "Database cannot be overwritten by restore." It's a pretty self-explanatory here, and we just go and set that on the Property tab on the properties of the database.

There are some considerations and restrictions to the Recovery Storage Group (slide 25). Only one Recovery Storage Group can be created on each server. Only the mailbox stores from the same admin group can be added to the Recovery Storage Group. After a mailbox store has been added, only mailboxes from the same original storage group can be added to the Recovery Storage Group, as I had shown earlier. An Exchange 2000 mailbox store can be added to the Recovery Storage Group, but must be Exchange 2000 SP3 or later.

Multiple mailbox stores can be added to the Recovery Storage Group. In order to have the store added to the Recovery Storage Group, it definitely must exist in Active Directory. All restores will be defaulting to the mailbox stores that are Recovery Storage Groups and not to the live databases.

We cannot use public folder stores or they cannot be added to the Recovery Storage Group (slide 26). New mailbox stores cannot be created in the Recovery Storage Group. You cannot use any other protocol other than MAPI to connect to the database here. No policies are applied. As far as the stores in Recovery Storage Group, the mailboxes will remain disconnected. New mailboxes cannot be created. I mentioned that already.

When you are using an active/passive cluster, you can create only one Recovery Storage Group per Exchange virtual server (slide 27). Mailbox stores from stand-alone servers can be added to the Recovery Storage Group on a cluster and vice versa, but they have to be in the same administrative group. You cannot back up the databases in the Recovery Storage Group. Online maintenance and defrag will not run on the Recovery databases. You cannot rename Recovery Storage Groups when they are created. Some of these restriction considerations are new here and some I had already mentioned in the earlier slides, but just wanted to recap here.

With this, I will really finish off my actual presentation and I will try to do a demonstration. We'll create a database, add it, create a Recovery Storage Group, add the databases, and see how it looks in the Exmerge. So now I will go through a demonstration and show you how to create the Recovery Storage Group and add databases to it (slide 28).

Once you go ahead and right-click your server object, click New, and you'll see that you have a new option called Recovery Storage Group here. Select Recovery Storage Group. We'll go ahead and modify the name a little. Click Apply. You'll see that as soon as the Apply is created, the log prefix changes to R, which is really for Recovery Storage Group.

Now the storage group has been added and we have multiple databases in our different storage groups. We'll go ahead and click Add a Database to Recover, just by right-clicking the Recovery Storage Group object. You will see databases from all the storage groups that we have on the server right now; we have five in total.

Let's say I select a database from the first storage group. I add it. The next time I go and add a database to recover, I will see only the databases from that storage group. Databases from other storage groups will not be available as an option. So you add all of them. Then we just go ahead and start our restore process.

When the restore is done, when you go to the restore set, you'll see that the public folder store is available in the restore set. But even if you select that entire set, the public folder database itself will not be restored because it's going to the Recovery Storage Group. When that is done, we will go ahead and mount the databases. At that time you will see that all the mailboxes are showing as disconnected, which is what is expected because it has to remain disconnected for the Recovery Storage Group.

This is what I wanted to show: how a Recovery Storage Group is created, how the databases are added, that the log file prefix changes, which are concepts I discussed earlier. With that, I will close my demonstration.

There are some good Knowledge Base articles about Recovery Storage Group, and there are some links where you can get other information. I know that there is also a detailed document in the works about the Recovery Storage Group. I don't have the exact online on it unfortunately, but I know that it will be available soon. So I would recommend that you keep checking the Exchange Web site for the latest editions to documentation about Exchange 2003, including the Recovery Storage Groups. Thank you very much.

Otto Cate: Great. Thank you very much for the presentation. We're going to go ahead and jump into the Q&A section here pretty soon. Before we roll over to the Q&A section, I'd just like to share a couple of real quick program notes with our audience here.

If you'd like to have a copy of the PowerPoint® slides, they are available for download through the site at support.microsoft.com/webcasts/. Simply click on today's event. Go to the Past Support WebCasts site and look under the Messaging section. This is also the same area where you'll find the on-demand streaming media archive.

Let's go ahead and jump into these questions here. Why are the only available storage groups and mailbox stores listed when you try to add a mail store back? Isn't it for recovery only? Shouldn't I pick a file instead of choices?

Irfan: As I mentioned, there are two ways you can do it. One is a restore from online backup. In this case, you have to add the databases from here. You can also add files directly, but still you have to add on a database-by-database basis because the way Exchange works, you really have to mount the database and attach them to really recover anything. It's not a flat file where we have other means of extracting the data.

Otto: The next question: What is the best way to implement and utilize VSC services on Microsoft Windows Server™ 2003 in recovery and backups of SGs and mail stores?

Irfan: I think I'll have to look into that and maybe address this question a little later. I may need a little detailed response there.

Otto: Great. The next question: If an employee leaves an organization and their account and mailbox have been deleted, and then six months or maybe a year later you realize that you need a group of messages that were in that employee's mailbox, for example about a key set of transactions with customers or something of that nature. Can that be recovered if you have Windows NT® backup tapes from that time period?

Irfan: That cannot be recovered via Recovery Storage Group because you have really removed all the traces from Active Directory, as we mentioned, if the account was deleted. If you had done the backup of Active Directory, as well as Exchange from that time, yes, but you will need a recovery storage server to prepare and do the restore. The answer is yes, but not via Recovery Storage Group.

Otto: Isn't it required to make the registry modification to enable Recovery Storage Groups, even after specifying an RSG in Exchange System Manager?

Irfan: No, it is not required to modify the registry to create the Recovery Storage Group by user. The only time when we need to modify the registry, within the concept of Recovery Storage Group, is when we want to temporarily override the presence of Recovery Storage Group, while it is still in the Exchange System Manager. That's the only time when we really need to go into the registry, as far as the Recovery Storage Group is concerned.

Otto: The next question here: You had mentioned that the only supported application is Exmerge. If MAPI is the exposed interface that Exmerge uses, how can I connect to the RSG via a MAPI interface like Exmerge?

Irfan: Yes. I mentioned that MAPI is the only interface that can be used to connect to the databases in Recovery Storage Group in specific circumstances because the mailboxes remain disconnected when the database is there. That functionality has been added to Exmerge only in the Exchange 2003 version, to do that. All you have to do is, after you have the Recovery Storage Group databases restored and mounted, you run Exmerge, point it to the server, and when you click Next, it will show you the list of available databases. You can pick which one you want and you'll see the Recovery Storage Group there. But, in general, any of the other normal MAPI application will not be able to connect, because of certain prerequisites that they have to go through that have been changed in Exmerge 2003.

Otto: The next question: I'm in the middle of a deployment consisting of a four-node cluster back end and then two front-end Exchange servers with two Internet Security and Acceleration (ISA) servers. We're looking at approximately 7,000 users. Could you please answer the question that we had just gone over? In that type of scenario, in that large an organization, if that affects what we had just discussed at all, or are we looking at the same troubleshooting for that kind of organization?

Irfan: Recovery Storage Group is really a scenario for recovering data on a database-by-database basis. So I don't think that has any bearing on how large or small the organization is, because we will be limiting the restore of a particular server in the Recovery Storage Group and just taking the data out. Yes, it is available in the clustered environment as well, as I mentioned. Hopefully, I've answered your question, but if that has not answered the question, you may want to pose another question giving more clarity.

Otto: Thanks for the clarification. The next question: Is there a way to enable the database to be overwritten by a restore option programmatically on the RSG?

Irfan: There could be, but this is not really an area of my expertise so I will not comment right now. I can follow up on this question.

Otto: I'm not sure if this is available in additional resources, but can you direct me to a disaster recovery white paper for Exchange 2003 in the event of a total server failure? I've been searching the site and haven't been able to find any specifics yet.

Irfan: I know that there's a paper in the works that talks about the Recovery Storage Group and how that can be used for disaster recovery. But as far as a total disaster server recovery or recovering from a total disaster of the server itself, I think all the principles of Exchange 2000 still apply. If you look at the Web site, there are documents available to that effect. Some of them are very much in-depth. What I will also do is I will look at the URLs, and possibly I will give it to Otto so that we can also put it in the transcript, so you can refer to it later on.

Follow-up answer: For total server failure, the process is similar to recovering Exchange Server 2000 and there is documentation available on the Web on this subject. The "Using Exchange 2003 Recovery Storage Groups" document is available for download at http://www.microsoft.com/downloads/details.aspx?FamilyID=df144af6-bee5-4b35-866a-557e25fe2ba1&DisplayLang=en.

Otto: Could you tell us a bit more about dial tone databases? It's actually a new term that I've never heard before.

Irfan: Let's say where we need it, so that will be easier to understand. Let's say you have a server with a database running, and the particular database experiences a corruption that requires you to restore. You know that the restore will take three to four hours. So one way, the way we used to do it in Exchange 2000 is that while we are doing the restore, users are unable to use the main server if we are doing the restore directory there. The dial tone concept that comes into play here is that you create a Recovery Storage Group; you delete the original database that has corruption (or you back it up or whatever). You create a new empty database, so that users can start using the database or e-mail functionality.

Then, in parallel, you can do the restore in the Recovery Storage Group. That way, during those two, three, or four hours that you are doing the actual restore, the user there is still able to use their e-mail by using this dial tone because it's a temporary database. Maybe it's more comprehensible where it is a temporary database, which we refer to as a dial tone. Once the restore has been completed, we can swap this restored database into the production section, and the dial tone or temporary database into a different area, and use Exmerge to take that differential three or four hours of data out of the dial tone temporary database and put it back into the production server.

The dial tone concept is basically a name for when we create a temporary database to give e-mail connectivity to users while the actual or larger database is being restored, repaired, or maintained, depending on the circumstances.

Otto: We have a couple of questions regarding public folders. We're wondering why public folders are not restored via RSG?

Irfan: On the public folder subject, I would just say that the functionality has not been supported within the Recovery Storage Group concept yet. Other than that, I would try to refrain from a public folder discussion because this is not really within the framework of today's presentation.

Otto: The next question: Can you use an RSG to recover a deleted AD user's mailbox, and then use Exmerge to create a .pst and then import it? If that's possible, why is RSG not supported when recovering a deleted AD user's mailbox?

Irfan: After the AD user has been deleted, no, you cannot use Recovery Storage Group because it looks at the same GUID as the mailbox within the database. If that is the case, then we will have to go through a recovery server. I also mentioned this a little bit in detail, so you should be able to get back to the scenarios and limitations of how you want to go about doing that in the transcript of today's presentation.

Otto: Under what scenarios would you use the dial tone method?

Irfan: One of the typical scenarios would be where you do hardware or for whatever reason your current database has become corrupted or is showing signs of some damage and you cannot, for example, back it up. So you would like to create a dial tone database, restore a previous database, and merge both the databases into one, so that you will get a complete database without the inherent problem, and you can continue doing the restores on an ongoing basis. This is one of the typical scenarios where you can use it.

Otto: If I include RSG and VSS in my backup strategy, can I use the integrated backup within 2003 for a complete backup strategy or do I still need a third-party backup package?

Irfan: Because the questions revolve around third-party packages or something that may or may not be within our focus, I would like to follow it up in the transcript, and we will post the answer to this later.

Otto: When you create an empty dial tone database, is there a mechanism to place a banner message into it explaining to the users that their mail gets restored?

Irfan: I don't believe there's a default option where, if you create a dial tone and the users log into this new database for the first time, they get something like, "You are not seeing the mail because recovery is being done," or something. The way to work around this would be when you are bringing the dial tone database up, just send an e-mail to all the affected users (maybe you have preconfigured distribution groups or you can select the users on a database or a server level). So the answer is no; I don't think we have any features like that.

Otto: We have a follow-up to one of the dial tone questions. The previous question was: Under what scenario would you use the dial tone method? The follow-up is: I understand how the dial tone database is created in Exchange 5.5, but is it different in Exchange 2003? If so, how is it created?

Irfan: The dial tone database is created just like you create any database within Exchange 2000 or 2003. Within the script, you just go ahead and create a database. You just need a different name so you can identify what the purpose of the database was. Essentially, the method is the similar, I would say.

I have just one thing to add on the dial tone subject. There are certain prerequisites or considerations you have to make when you are creating the dial tone in order to save your current file, and I would recommend that you do. It's a little bit more in-depth than what we can cover here, but I just wanted to highlight so that you are aware of it. There is some more information about it that you may want to look at before going into this area. You can see the documents that are available at http://www.microsoft.com/exchange/.

Otto: The next question: Are there any plans to make the MAPI prerequisites available to use a standard MAPI interface on an RSG, or do you know if this is going to remain essentially as such? I'm not sure if that's something that we know off the top of our heads.

Irfan: For the time being, I think I can really safely say that Exmerge is the way to go. As far as the future plans are concerned, I don't think I'll be able to comment on that.

Otto: The next question here: Is it possible to use Microsoft Outlook® 2003 Cache mode to restore data back to an empty database in the case of a disaster, if the sync is set to 15 minutes, allowing only the loss of 15 minutes worth of data?

Irfan: Again, this question is a little bit outside my area of expertise about the functionality of Outlook. I will follow it up.

Otto: With that, it does appear that we've answered all the questions that were submitted to the queue today. So I'm going to go ahead and wrap up our session here. I certainly wanted to thank our presenter for coming out and giving us a great presentation. Of course, as always, I wanted to thank our audience for coming out and listening to this presentation. We certainly hope that this information was helpful to you and your business. We hope that everyone has the opportunity to tune in again soon. Thanks, and have a great day.