To understand the methods that are described in this article, you must understand something about the building blocks of an Exchange Server mailbox database.
An Exchange Server database file is a set of 4-kilobyte (KB) pages, numbered in sequence. The first two pages of the database are the database header. The third page is logical page number 1, the fourth page in the database is logical page number 2, and so on.
Pages are organized in balanced trees (b+ trees). These trees are collections of pages that are linked together logically for fast search and retrieval of information.
Trees are organized in tables. A database table is typically made up of several trees that serve specialized functions in the table. There is a data tree, and there may also be index and long-value trees, along with other trees for specialized system functions. (A long value tree is used to hold data that cannot fit in its entirety on a single data page.)
Tables are organized in mailboxes. Each folder in a mailbox is a separate table. Along with tables that make up mailboxes, there are tables shared between mailboxes, such as the folders, messages, and attachments tables. The Mailbox table lists all the mailboxes in the database.
There are also several system tables in the database. The two most important system tables are the catalog (the MSysObjects table) and a second copy of the catalog (MsysObjectsShadow). These catalogs are the "table of tables." If they are both lost, the database does not know what other tables it contains, or where to find the other tables.
How the Eseutil Utility Repairs Database Files
Eseutil.exe is a multifunction utility that is included with Exchange Server 5.5 and Exchange 2000 Server. You can use the Eseutil utility to view database information, defragment the database file, and perform several other functions, which include repair of a damaged database.
In repair mode, the Eseutil utility fixes individual tables. The Eseutil utility does not maintain the relationships between tables. For example, if the Msgs table, which contains all messages for all mailboxes, is damaged, when the Eseutil utility repairs the Msgs table, the Eseutil utility may need to discard some unreadable messages. These messages may still be listed as valid in some users' Inboxes. To resolve this problem, you have to run the Isinteg.exe utility, which is also included with Exchange Server. This utility maintains the relationship between the tables, and removes any discarded items from the user's list of messages in the Inbox.
Before you run a repair procedure, it is useful to know what the chances are that the repair will succeed, and where data is likely to be lost. When you run repair there is always a chance of catastrophic data loss, and you should always have a backup of a database file before you attempt repair. It is safer to restore data from an online backup and roll forward through the transaction logs to bring the database completely up to date than it is to run repair. Use repair when you cannot follow this preferred strategy for some reason.
To discover the purpose of a given page in the database, you need to use both the Eseutil and Isinteg utilities together. For Isinteg to gather the necessary information, the database must be startable, and the database must be installed on a functioning Exchange Server computer, because Isinteg actually starts the database silently to read the database as a collection of mailboxes, not just a collection of tables.
The methods in this article apply to both Exchange Server 5.5 and to Exchange 2000 Server. The database and page structures in both versions are similar; you can use the version of Eseutil that is included with Exchange 2000 Server in some read-only modes against an Exchange Server 5.5 database. Microsoft recommends that you run the Exchange 2000 Server version of Eseutil against Exchange Server 5.5 databases in read-only modes.WARNING
: Do not use a mismatched version of Eseutil in a mode that causes changes to the database. These modes include, but are not limited to, use of the /r
, and /d
command-line switches. The only mode that is definitely safe is the /m
mode, which dumps various types of information but does not alter the database.
Be absolutely certain that you know which version of Eseutil you are running at all times. If you run Eseutil with no switches, the version information is displayed on screen, and no database operations are performed.
To run the Exchange 2000 Server version of Eseutil against an Exchange Server 5.5 database, copy the Eseutil.exe and Ese.dll files from an Exchange 2000 Server computer to a convenient location together, start Eseutil from that folder, and specify the path to the database file.
The rest of this article describes a typical scenario in which a damaged page is discovered in a database and that page is resolved to a specific folder.
In this scenario, the following event is logged in the application event log on an Exchange Server 5.5 computer:
Event Type: Error
Event Source: ESE97
Event Category: Database Page Cache
Event ID: 200
Time: 3:39:05 PM
MSExchangeIS (2312) A read of the database file e:\exchsrvr\MDBDATA\PRIV.EDB between offsets 0x0000000000212000 and 0x0000000000212FFF failed after 16 failed read attempts with error -1018. There is a software or hardware problem affecting the database drive that must be corrected to preserve database integrity. Contact Microsoft Product Support Services.
Determining Which Mailbox Owns a Database Page
This section contains the following information:
- How to Determine Which Database Page Is Damaged
- How to Determine Which Tree Owns a Page
- How to Determine Which Table Owns a Tree
- How to Determine Which Mailbox Owns a Table
How to Determine Which Database Page Is Damaged
Use one of the following methods to determine which page in the database is damaged:
- Calculate the page number from the offsets in the event error message. In this example, the damaged page is 0x212000 bytes into the database. In decimal numbers, this number is 2,170,880. Divide that number by 4,096 and subtract 1 to determine the number of the damaged page, which in this example is 529. -or-
- Use the Esefile utility to perform the calculation for you by scanning an entire database and reporting every page that has physical damage.
: You can use the Calc.exe utility in scientific mode to easily convert hexadecimal and decimal numbers.
After you determine that page 529 is damaged, you need to determine which tree the page belongs to.
How to Determine Which Tree Owns a Page
The Exchange 2000 version of the Eseutil utility introduces the capability to dump the logical information in the header of any page in the database. This capability works also against Exchange Server 5.5 databases. To dump a page, run the following command:
eseutil /m [database_name] /pnnnnn > file_name.txt
is the name of the database, nnnnn
is the number of the page that is damaged, and file_name
is the name that you want to give the output file.
For example, if the database name is Priv1.edb, the page number of the damaged page is 529, and you want to save the output in the plain text file 529.txt, run the following command:
eseutil /m priv1.edb /p529 > 529.txt
The beginning of the resulting page dump is similar to the following:
Microsoft(R) Exchange Server(TM) Database UtilitiesVersion 6.0Copyright (C) Microsoft Corporation 1991-2000. All Rights Reserved.Initiating FILE DUMP mode...Database: priv.edbPage: 529pgnoThis <0x02660004 , 4>: 529 (0x00000211)objidFDP <0x02660018 , 4>: 123 (0x0000007b)ulChecksumParity <0x02660000 , 4>: 881043383 (0x3483a7b7) ** computed checksum: 2653031709 (0x9e220d1d)dbtimeDirtied <0x02660008 , 8>: 522872 (0x000000000007fa78)cbFree <0x0266001c , 2>: 4026 (0x0fba)ibMicFree <0x02660020 , 2>: 1675 (0x068b)itagMicFree <0x02660022 , 2>: 2 (0x0002)cbUncommittedFree <0x0266001e , 2>: 0 (0x0000)pgnoNext <0x02660014 , 4>: 0 (0x00000000)pgnoPrev <0x02660010 , 4>: 0 (0x00000000)fFlags <0x02660024 , 4>: 1 (0x00000001)Parent of leafInternal pageRoot pageFDP pageMultiple Extent Space (ParentFDP: 1, pgnoOE: 2341)Primary page
Locate the line in this output that begins with objidFDP, which in this example has a number of 123. Each tree in the database is identified uniquely by an objidFDP number. If you dump all of the single pages in the database and collect all the pages that have an objidFDP of 123, those pages compose that particular tree in its entirety.
Even though the page is damaged, the page dump utility reads whatever information is left intact. If the page is too damaged to obtain a readable header dump, you can try to dump the surrounding pages. If the pages that precede and follow the damaged page have objidFDP values that match each other, the damaged page is also likely to be part of the same tree.NOTE
: If you want to dump all the pages in a database at once, you can do it with a command similar to the following; note that this process may take a long time for a large database
for /l %a in (1,1,nnnnn) do eseutil /m priv1.edb /p%a >>allpages.txt
is the number of pages in the database. To determine the number of pages, divide the exact byte size of the database by 4096, and then subtract 2 (for the database header). For example, if the database is 56,631,296 bytes in size, it has 13,824 pages.
After you determine that the page belongs to tree 123, you need to determine which table owns tree 123.
How to Determine Which Table Owns a Tree
You can determine which table owns a tree by using the Exchange 2000 version of the Eseutil utility to dump the space usage in the database. A space dump lists each table in the database, and lists how many pages each table is using. The Eseutil utility for Exchange Server 5.5 can also dump space usage, but the Exchange 2000 version adds a column to the output that identifies the objidFDP value for each tree in each table.
To use the Exchange 2000 version of the Eseutil utility against either an Exchange Server 5.5 or Exchange 2000 database to dump the space usage, run the following command:
eseutil /ms priv1.edb >space.txt
The output is similar to the following (irrelevant columns have been abbreviated for this example):
Microsoft(R) Exchange Server(TM) Database UtilitiesVersion 6.0Copyright (C) Microsoft Corporation 1991-2000. All Rights Reserved. Initiating FILE DUMP mode...Database: priv.edb******************* SPACE DUMP ********************Name Type ObjidFDP PgnoFDP ===================================================priv.edb Db 1 1 1-1511 Tbl 98 433 MsgFolderIndex7 Idx 100 434 MsgFolderIndexPtagDel Idx 102 436 RuleMsgFolderIndex Idx 101 435 1-172E Tbl 103 457 MsgFolderIndex7 Idx 105 458 MsgFolderIndexPtagDel Idx 107 460 RuleMsgFolderIndex Idx 106 459 1-194E Tbl 108 465 MsgFolderIndex7 Idx 110 466 MsgFolderIndexPtagDel Idx 112 468 RuleMsgFolderIndex Idx 111 467 1-1B6D Tbl 113 489 MsgFolderIndex7 Idx 115 490 MsgFolderIndexPtagDel Idx 117 492 RuleMsgFolderIndex Idx 116 491 1-1D8A Tbl 118 502 MsgFolderIndex7 Idx 120 503 MsgFolderIndexPtagDel Idx 122 505 RuleMsgFolderIndex Idx 121 504 1-1FA8 Tbl 123 529 <- tree 123 MsgFolderIndex7 Idx 125 530 MsgFolderIndexPtagDel Idx 127 532 RuleMsgFolderIndex Idx 126 531
This output indicates that the tree with the objidFDP of 123 is part of the data tree for table 1-1FA8.HINT
: When you search for the objidFDP number in a space dump, use a space before and after the number, rather than searching for only the number, to reduce multiple matches. For this example, search for " 123 " (without the quotation marks).
Tables in an Exchange Server database are identified by either a hyphenated number (such as 1-1FA8) or by a name (such as Msg). Many system tables are identified by name, but not all of them are. For example, table 1-24 holds all attachments in the database. All user mailbox folder tables are numbered, not named.
In this example, the page number, 529, is also contained in the space dump output. However, this does not mean that you do not need to dump the page to find the tree number. You cannot go directly to the space dump and look for the page.
The page that is listed in the space dump is the root page for the tree. Page 529 happens to be the root page of the tree in this example, but it might not have been, and in that case page 529 would not be listed in the space dump.
After you determine that page 529 belongs to tree 123, which belongs to table 1-1FA8, you need to determine which mailbox owns the table. If page 529 belongs to the Msgs table, that indicates that there is some kind of damage to at least one message in the database. However, because the table that is involved is not named, you need to determine what type of table 1-1FA8 is.
How to Find the Mailbox That Owns a Table
If the page is not in a system or shared table (such as 1-24 or Msg), the page probably belongs to a particular mailbox. Isinteg.exe allows you to dump all folders, and identifies the type of the folders. Therefore, you can usually tell that a table is a Sent Items or Inbox folder. In with Exchange Server 5.5 Service Pack 3 and later, Isinteg also dumps the mailbox table, which enables you to correlate a particular folder with the mailbox that owns it. To dump the folder and mailbox tables, run the following command with the version of the Isinteg.exe file that is included with Exchange Server 5.5 Service Pack 3 or later:
isinteg -pri -dump -l isdump.txt
If you are running this command on Exchange 2000, run the following command:
isinteg -s <servername> -dump -l c:\dump1.txt
The Isinteg dump file is likely to be very large. The file contains three major sections: the Index Age table, the Folder table, and the Mailbox table. Find the table that you identified from the Eseutil space dump in the Folder table section of the Isinteg dump. The following is the Folder table entry in the Isdump.txt file for 1-1FA8:
 Folder FID=0001-000000001FA8 Parent FID=0001-000000001FA5 Root FID=0001-000000001FA4 ACL ID=0000-000000000000 Folder Type=1 Msg Count=859 Msgs Unread=859 Msgs Submitted=0 Rcv Count=4 Subfolders=0 Name=Inbox Comment= Restriction= Search FIDs= Recursive FIDs= Search Backlinks= Categ FIDs=
: The easiest way to search for the folder 1-1FA8 is to simply search for the suffix 1FA8. In the Isinteg output, the folder ID (FID) of the table is padded with zeroes to make all IDs the same number of digits, which makes it impossible to search for 1-1FA8.
Notice that the Name value for the table is Inbox, which indicates that the bad page is a user's Inbox. In this example, the root FID that is listed for 1-1FA8 is 1-1FA4 (or 0001-000000001FA4). After you determine the root FID, you can search the Mailbox table.
The Mailbox table is at the bottom of the dump. The easiest way to find the beginning of the Mailbox table is to search for the following words:
Dump of the
When you search the Mailbox table, you will see the following:
 RootFID=0001-000000001FA4 Owner DN=??? GUID=A5C4D99B DE518E40 82BD51B8 197571A7 Display Name=David Johnson Comment= Sentmail FID=542D-6530362B5136 Subtree=0001-000000001FA5 Inbox=0001-000000001FA6 Outbox=0001-000000001FA7 Sentmail=0001-000000001FA8 Finder=0001-000000001FAA DAF=0001-000000001FAB Spooler Q=0001-000000001FAC Size=(ec:ecNotFound-MAPI_E_NOT_FOUND) Localized=TRUE Locale=0x0Error: 1 database retrieve error on current record.
Ignore the error at the end of the mailbox record; this error is an anomaly of the Mailbox dump output and is always displayed.
This Mailbox table indicates that damage was done to David Johnson's Inbox folder.
Contact the user and ask the user to start Microsoft Outlook. In this example, David's Inbox says he has 968 messages, but David cannot see a single one of them.
Assessing the Damage
The page dump in the "How to Determine Which Tree Owns a Page" section of this article lists several "flags" on the page that indicate the nature of what is stored on the page.
These flags were:
- Parent of leaf
- Internal page
- Root page
- FDP page
- Multiple Extent Space (ParentFDP: 1, pgnoOE: 2341)
- Primary page
There are several other page types or flags that can be set on a page, but the only two that you need to be concerned about are the "Internal page" and "Leaf page" flags.
Internal pages are pages that help define the structure of each tree in the database. Leaf pages hold the actual data in the tree.
In most cases, even if all of the single internal pages for a tree are lost, the tree can still be reconstructed with perfect fidelity by the Eseutil utility repair function, because each leaf page knows which tree it belongs to, and the relationships between the pages can be recalculated.
But if a leaf page is lost, some data is always lost, unless the leaf is from one of the system catalog tables. In that case, you can retrieve the page from the other copy of the catalog.
In this example, even though David cannot see any of the messages, the damage is only to a single, internal page in the tree, therefore it is likely that repair will succeed completely.
Only undertake repair with the eseutil /p
command on a copy of the database, not the original database. No matter how positive all indications are, repair always carries the risk of permanent data loss.
After you run the eseutil /p priv.edb /v /x
command on this database, all of the user's messages are visible again. Because the damage was to an internal page and it is very likely that repair was 100% successful, you do not need to run Isinteg afterward. If a leaf page in the user's Inbox folder was damaged, you would need to run Isinteg to restore the integrity of the relationship between the user's Inbox and the shared data tables in the database. Refer to the Exchange Server documentation about Isinteg to determine which Isinteg switches and tests you need. You can also run Isinteg in an "alltests" mode for a comprehensive check and fix.
- Eseutil can perform an integrity check of the database, which is more or less a "practice run" of repair. The eseutil /g command may stop working before Eseutil reaches the end of the database if the database is severely damaged or if one of the system catalog tables is damaged. If one of the MSysObjects tables is not intact, you must run repair to restore integrity to the catalog before you can check any of the other tables. In such a case, you cannot predict in advance of running repair which tables or pages are damaged.
- The last table that the integrity checker or repair traverses is the TimedEvents table. If you do not see this table at the bottom of the Eseutil output in verbose mode (eseutil /g priv.edb /v /x), then the integrity check did not finish.
NOTE: In Exchange 2000, the /v (verbose) and /x (extended) switches are implied and are no longer needed. Redirect the output of the integrity check to a file (eseutil /g >file.txt) so that you can examine the results. You can check the progress of the integrity check periodically by examining the file in a text editor.
- By far the two largest tables in a typical database are the messages (Msg) and attachments (1-24) tables. Therefore, these tables are statistically more likely than others to be damaged because they comprise so much of the database. If you perform repair when either of these tables is damaged, you risk losing all the data in these tables.
- You are much more likely to suffer damage to a leaf page than to an interior page because most pages in the database are leaf pages.
- If damage has been done to system tables, such as the catalog, or other tables that do not contain user data, it is risky to leave a repaired database in service. The risk from this is relatively low, but it exists. Even if no problems are immediately noticed, you may encounter problems such as high CPU usage, or sudden stops of the database as these tables are accessed. When system tables have been damaged, implement a plan to salvage data and discard the database as soon as feasible. The easiest way to do this is to move all the mailboxes to another server, delete the damaged database, and then move the mailboxes back to a fresh database.