DOCUMENT:Q272570 06-AUG-2002 [exchange] TITLE :XADM: Disaster Recovery of Information Store on Exchange Server PRODUCT :Microsoft Exchange PROD/VER::4.5,5.5 OPER/SYS: KEYWORDS:kberrmsg exc55 ====================================================================== ------------------------------------------------------------------------------- The information in this article applies to: - Microsoft Exchange Server, version 5.5 - Microsoft BackOffice Small Business Server version 4.5 ------------------------------------------------------------------------------- SUMMARY ======= This article describes how to recover from information store corruption by using an offline restore procedure of Microsoft Exchange Server data. NOTE: This article does not describe disaster recovery methods for any other areas of Exchange Server, and the solution described in this article may not correct all instances of information store corruption. MORE INFORMATION ================ When corruption occurs in Exchange Server 5.5, one of the most commonly affected components is the information store. Corruption usually involves any one of the .edb (Exchange Server database) files. In most cases, either the Priv.edb file or the Pub.edb file (located in the Exchsrvr\Mdbdata folder) is corrupted. When the information store is corrupted, the following symptoms occur: - The Microsoft Exchange Server information store service does not start. - The information store cannot be stopped. - The information store stops responding and the CPU usage level remains at 100 percent. - Clients are unable to retrieve or send e-mail. This behavior does not change if you stop the information store, and then restart it. - When you try to restore the Exchange Server databases from an offline backup, and then run the Information Store Integrity utility (Isinteg) patch, you receive the following error message: Error 4294966746: JET_errDatabaseInconsistent The procedure described in this article is based on a single-server, single-site scenario with the following qualities: - A site connector is not being used. - An offline copy of both the Pub.edb and Priv.edb files exists. - A Key Management Server is not being used. - The server is running Microsoft Small Business Server 4.5. However, the principles can apply to most other Exchange Server 5.5 computers that are set up in a single-organization, single-site configuration. NOTE: This setup is not designed to restore connector settings. Troubleshooting --------------- Use the following procedure to isolate information store corruption, and to help you recover the public and private information stores. Before you start, make sure that you can recover the data files from another source. For example, if you are using a small Exchange Server database (less than 1 gigabyte [GB]), make a copy of the Pub.edb and Priv.edb files. For larger databases, if you do not have enough disk space to copy the files, use a good tape backup system or another offline storage utility. To troubleshoot the information store: 1. Try to start the information store. Record any error and event IDs that are reported. 2. Check the consistency of the databases. If a database is consistent, all the log files have been committed to the store. If the database is not consistent, the database may not be corrupted; the log files might not have been committed to the database yet. Run the following commands to check the consistency of the public and private databases: "c:\exchsrvr\bin>eseutil /mh ..\mdbdata\priv.edb" (without the quotation marks) "c:\exchsrvr\bin>eseutil /mh ..\mdbdata\pub.edb" (without the quotation marks) NOTE: This example assumes that you are running the programs from the ExchSrvr\Bin folder, and that the .edb files are in the same ExchSrvr folder in the Mdbdata folder. To verify that the .edb file is consistent, check the output line labeled "State." If you would like to send the output of this command directly to a text file, you can modify the command line as follows: "c:\exchsrvr\bin>eseutil /mh ..\mdbdata\priv.edb > .txt" (without the quotation marks) "c:\exchsrvr\bin>eseutil /mh ..\mdbdata\pub.edb > .txt" (without the quotation marks) To have the text displayed directly on the screen, add "| more" (without the quotation marks) to the end of the command. If Pub.edb and Priv.edb are not located in the ExchSrvr\Mdbdata folder on the same drive as the ExchSrvr\Bin folder that you are running the command from, replace the "..\mdbdata\priv.edb" portion of the command with the full path of the Priv.edb file. Make the same changes for Pub.edb. If both the Pub.edb and Priv.edb files are consistent, go directly to step six. If one of the databases is inconsistent, attempt a soft recovery of the files to check if there are copies of the log files stored in the Mdbdata folder. If you do not have the log files, you are not able to complete the soft recovery process. When you do a soft recovery, any uncommitted log files are then committed to the information store database. To initiate a soft recovery, run the following commands. The first command is for the private information store. The second command is for the public information store: "c:\exchsrvr\bin>eseutil /r /ispriv" (without the quotation marks) "c:\exchsrvr\bin>eseutil /r /ispub" (without the quotation marks) This command enables you to recover the databases (if the databases are fully recoverable). To confirm this behavior, repeat this step from the beginning, and then run the "eseutil /mh" (without the quotation marks) command again. If the state has changed to "Consistent" for both the public and private information store, then go directly to step six. 3. Defragment the file that you just ran the recovery on. You must do this procedure if you are running short of disk space. If not, this step may not be necessary. If you ran the soft recovery procedure on both Pub.edb and Priv.edb, then run the following defragmentation commands on both files: - To defragment Priv.edb: "c:\exchsrvr\bin>eseutil /d /ispriv" (without the quotation marks) - To defragment Pub.edb: "c:\exchsrvr\bin>eseutil /d /ispub" (without the quotation marks) NOTE: If the private and public information store databases are large, this step may take some time. If you receive an error message that indicates a lack of space when you attempt this step, copy the ExchSrvr\Bin folder to a drive with more space and run the command from that location. For additional information about how to resolve this error message, click the article number below to view the article in the Microsoft Knowledge Base: Q192185 XADM: How to Defragment with the Eseutil Utility (Eseutil.exe) 4. If either of the databases are inconsistent, and the eseutil /r command does not return them to a consistent state, use the following procedure: NOTE: When you use the following command, you may lose some Exchange Server data, including the messages that were contained in the log files that were not yet committed to the information store database. The following command is a hard or forcible state recovery command, and should only be used if the public or private information stores did not return to a consistent state after you followed the previous steps. At the command prompt, type the following commands: - To bring the Priv.edb file back into a consistent state, type the following command at a command prompt: "c:\exchsrvr\bin>eseutil /p /ispriv" (without the quotation marks) - To bring the Pub.edb file back into a consistent state, type the following command at a command prompt: "c:\exchsrvr\bin>eseutil /p /ispub" (without the quotation marks) 5. Delete all .log files in the Mdbdata folder as well as the .chk file, and the Temp.edb file (if it exists). 6. Use Isinteg.exe to fix Pub.edb and Priv.edb. This utility runs a test on all areas of each of the databases and reports the results. It also attempts to fix any issues that are encountered. - To fix Priv.edb, type the following command at a command prompt: "c:\exchsrvr\bin>isinteg -pri -fix -test alltests" (without the quotation marks) - To fix Pub.edb, type the following command at a command prompt: "c:\exchsrvr\bin>isinteg -pub -fix -test alltests" (without the quotation marks) You receive a summary report which indicates the number of tests that were run, the number of warnings, the number of errors found, the number of fixes attempted, and the time it took for the program to run. If there are any warnings, errors, or fixes, run the command again. Repeat this step until there are no warnings, errors, or fixes. NOTE: In some cases, the same warnings, errors, or fixes are reported every time you run the program. If you receive the same warnings, errors, or fixes three times in a row, move to the next step. The errors may be caused by a corrupted e-mail attachment or something similar, which does not prevent you from starting the information store. If you can start the information store, and if it is stable, if errors and warning messages continue to be reported after you have run Isinteg.exe several times, use the ExMerge utility to repair the information store. The directions to use the ExMerge utility are listed later in this article. 7. Attempt to start the Information Store Database. If the Information Store Starts, proceed to the next step. If the Information Store fails to start with an error -1011 you will need to bring the Pub.edb and Priv.edb back to a state where the information store can recognize them. To do this, run the following command for both the public and private information store: "c:\exchsrvr\bin>isinteg -patch" (without the quotation marks) NOTE: The Microsoft Exchange directory service must be started before you run this command. This command works for both the public and the private databases. This command works for both the public and the private databases. You do not need to complete this step if you have already performed an online restore of the data. However, if you receive an -1011 error message when you try to start the information store, complete this step. 8. Restart the information store. 9. Run the Exchange Server Consistency Adjuster. The directory service/information store (DS/IS) Consistency Adjuster is used in disaster recovery scenarios where it is not possible to restore the folder to the recovery computer, or where the information store (the Priv.edb or the Pub.edb file) is copied to a recovery computer with a name that is different from that of the original server. If you did not lose the directory service or you did not reinstall Exchange Server with a clean folder, then you would not need to run this tool. NOTE: If you use this utility if you have multiple sites in your Exchange Server organization or multiple servers in your Exchange Server site, public folders may be rehomed unexpectedly. For additional information about the DS/IS Consistency Adjuster, click the article number below to view the article in the Microsoft Knowledge Base: Q182979 XADM: Function and Effects of the DS/IS Consistency Adjuster Using the Consistency Adjuster ------------------------------ To run the Consistency Adjuster, following these steps: 1. Start the Exchange Server Administrator program. 2. Open the container for your server. 3. Open the configuration container. 4. Open the Servers container. 5. Click your server. 6. On the File menu, click Properties, and then click the Advanced tab. 7. Click Consistency Adjuster, select all of the Operations check boxes, and then click OK. The Consistency Adjuster runs on both the public and private information stores. If you are performing this recovery on another server, or if you removed and reinstalled Exchange Server on this server and do not have a working copy of the directory store (Dir.edb), then there are two methods for recovering the users mailboxes. For both these methods, you are required to re-create any distribution lists and membership to those distribution lists that were lost. Method One: If you are able to access the user accounts, start the Exchange Server Administrator program, and then click "Extract Windows NT Account List" on the Tools menu. Make sure that you extract the user list from the domain, and not from the local computer. This procedure enables you to create a file that contains a list of all of the Windows NT users. After the file has been created, on the Tools menu, click Directory Import. When you import this list, all of the recipient mailboxes are restored. You may want to run the Consistency Adjuster again after this step. Method Two: If you do not have access to the Windows NT user accounts for these recipients, you must manually re-create all of the Windows NT user accounts with the same user names. Then, start the Exchange Server Administrator program, and then click "Extract Windows NT Account List" on the Tools menu. Make sure that you extract the user list from the domain, and not from the local computer. This procedure enables you to create a file that contains a list of all of the Windows NT users. After the file has been created, on the Tools menu, click Directory Import. When you import this list, all of the recipients? mailboxes are restored. After you complete either of these methods, if the mailbox has not been restored, you may have created a user account that has a name that is different from the previous Exchange Server recipient. At this point, the server should be functioning, and users should be able to retrieve e-mail. Using the ExMerge Utility ------------------------- If the information store starts, but there are other issues, use the ExMerge utility to export all of the user's mailboxes to .pst files. Then, merge the .pst files to user mailboxes in a newly created Priv.edb file. If you use offline storage for mail (.ost files), you need to move the .ost files to a .pst file before you start this procedure to prevent you from losing any .ost data. Before you use the ExMerge utility, be aware that ExMerge enables you to move user mailboxes only. You cannot use ExMerge to move specific Exchange Server configuration properties such as connector settings, Outlook settings, or client-side rules. You cannot use ExMerge.exe to move distribution lists, memberships in those distribution lists, or public folders. If you attempted this procedure, you must re-create or reconfigure the distributions lists manually. This procedure also destroys your single instance store for messages up to that point. As a result, the Exchange Server databases run the risk of growing substantially. For additional information about the ExMerge utility, click the article number below to view the article in the Microsoft Knowledge Base: Q174197 XADM: Microsoft Exchange Mailbox Merge Program (Exmerge.exe) Information To download the ExMerge utility, visit the following Microsoft Web site: http://download.microsoft.com/download/exch55/utility/q224493/nt4alpha/en-us/iloveyouhlpi.zip (http://download.microsoft.com/download/exch55/utility/q224493/nt4alpha/en-us/iloveyouhlpi.zip) NOTE: This version of ExMerge was publicly used to deal with the "Love Bug" virus. This is why the download is named Iloveyouhlpi.zip. After you have downloaded and extracted the ExMerge utility, delete the Exmerge.ini file that was placed in the extract folder for ExMerge. This procedure prevents errors from occurring while you are running the ExMerge utility. Next, follow these steps: 1. Make sure no users are currently accessing e-mail or have their e-mail client programs running. 2. Run ExMerge using step one of the two-step merge, and then enter all users. 3. When ExMerge is finished, stop the Exchange Server information store service, and then go into the ExchSrvr\Mdbdata folder and rename the Pub.edb and Priv.edb files. Delete all other files in this folder. Make sure that you do not delete the renamed Pub.edb and Priv.edb files until after you are sure the merge completed successfully. 4. Restart the Exchange Server information store service. 5. Start Outlook on a client computer, and then send an e-mail message to all of the users in the Exchange Server global address book. NOTE: If you do not complete this step, the second step of the ExMerge two-step merge does not show that there are any available mailboxes to restore. 6. Run ExMerge using step two of the two-step merge, and then enter all users. When this process is complete, the e-mail that was in the .pst files are moved to Exchange Server mailboxes. You should now have a new Priv.edb file that contains fresh copies of all mail messages. For additional information about restoring the Exchange Server information store, click the article numbers below to view the articles in the Microsoft Knowledge Base: Q231299 XADM: Information Store Returns Service-Specific Error 335544375 Q162354 XADM: DS_E_COMMUNICATIONS_PROBLEM Running ISINTEG -PATCH Q172813 XADM: Troubleshooting High CPU Utilization By Store.exe Q242364 XADM: Information Store Does Not Start After an Offline Defragmentation With 4294966277 Q237767 XADM: Making and Restoring Offline Backups Q219419 XADM: Information Store Stops Unexpectedly and Cannot Be Repaired For more information about Exchange Server disaster recovery, refer to the following white paper that is located at the following Microsoft Web site: http://www.microsoft.com/exchange/techinfo/administration/55/BackupRestore.asp For more information about the Isinteg.exe and Eseutil.exe utilities, refer to the documents that are located at the following path on the Exchange Server CD-ROM: \ExchSrvr\Server\Support\Utils The files are called Eseutil.rtf and Isinteg.rtf Additional query words: IS, smallbiz, SBS GAB .edb pub.edb priv.edb .ost .log isinteg.exe and eseutil.exe ====================================================================== Keywords : kberrmsg exc55 Technology : kbAudDeveloper kbExchangeSearch kbExchange550 kbZNotKeyword2 kbSBServSearch kbSBServ450 Version : :4.5,5.5 Issue type : kbhowto ============================================================================= THE INFORMATION PROVIDED IN THE MICROSOFT KNOWLEDGE BASE IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND. MICROSOFT DISCLAIMS ALL WARRANTIES, EITHER EXPRESS OR IMPLIED, INCLUDING THE WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. IN NO EVENT SHALL MICROSOFT CORPORATION OR ITS SUPPLIERS BE LIABLE FOR ANY DAMAGES WHATSOEVER INCLUDING DIRECT, INDIRECT, INCIDENTAL, CONSEQUENTIAL, LOSS OF BUSINESS PROFITS OR SPECIAL DAMAGES, EVEN IF MICROSOFT CORPORATION OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. SOME STATES DO NOT ALLOW THE EXCLUSION OR LIMITATION OF LIABILITY FOR CONSEQUENTIAL OR INCIDENTAL DAMAGES SO THE FOREGOING LIMITATION MAY NOT APPLY. Copyright Microsoft Corporation 2002.