XADM: How Circular Logging Affects the Use of Transaction Log

• Microsoft Exchange Server, version 4.0

SUMMARY

Microsoft Exchange Server stores its data in Jet databases. To commit this data to the database, Microsoft Exchange writes each transaction to a transaction log. In the background, these transaction logs are committed to the database. Microsoft Exchange Server has a circular logging option that, when enabled, limits the amount of disk space used by these transaction logs. This article describes the ways in which circular logging affects the use of transaction logs in Microsoft Exchange.

MORE INFORMATION

Transaction Logs and How Microsoft Exchange Uses Them

Transaction logs are files that Microsoft Exchange uses to commit data (e- mail messages, user additions, creation of folders, and so forth) to the corresponding database file on disk. For example in the \Exchsrvr\Mdbdata directory you may see files such as Edb.log, Edb00001.log, Priv.edb, and Pub.edb. The Edb*.log files are the transaction logs and the *.edb files are the information store database files. (Priv.edb is the private information store and Pub.edb is the public information store.) The most current transactions are stored in Edb.log; when Edb.log reaches 5 MB in size, a file called Edbtmp.log is created to accept incoming transactions and Edb.log in renamed to Edb00001.log. After Edb.log is renamed, Edbtmp.log is then renamed Edb.log. This process is repeated each time Edb.log reaches 5 MB in size. Thus, the log files build up as more and more transactions are created.

Microsoft Exchange Server uses these write-ahead transaction logs to ensure up-to-the-second recoverability of the database. Each (operation) transaction is written to a transaction file and to the database cache, and eventually to the database. After data is written to the database, a checkpoint (Edb.chk) is advanced. This checkpoint marks the position in the log files at which the database is in a consistent state. Thus, within the set of transaction logs there is a portion that can be considered active and a portion that can be considered inactive. If an event such as a power outage causes the system to fail, Microsoft Exchange goes into automatic recovery when the system is restarted. This entails rolling forward through the transactions in the existing log files to the checkpoint.

As described earlier, during recovery, the inactive portion of each log is not used; however, the logs can prove to be valuable when media failures occur. If the transaction logs are hosted on a different medium than the database and the medium containing the database fails, it is possible to generate a complete recovery by restoring the last full backup on the new medium and then roll forward through all the log files. Essentially, the checkpoint gets moved back from its location relative to the database on the disk that failed. Media failure is not a common scenario, but this is the only way (other than a fault tolerant solution, such as mirroring) that Microsoft Exchange can recover from media failures.

When circular logging is disabled, the system will continue to create log files until a backup is completed. Backing up Microsoft Exchange Server is the preferred way of saving the log files and removing these logs from the disk to free up space.

For additional information on freeing up disk space, please see the following article in the Microsoft Knowledge Base:

   ARTICLE-ID: Q128325
   TITLE     : Reclaiming Disk Space for the Information Store

Effect of Having Circular Logging Turned On (Setup Default)

When circular logging is on, Microsoft Exchange writes transaction logs as usual; however, after the checkpoint (Edb.chk) file has been advanced, the inactive portion of the transaction log is discarded and overwritten. Typically, this represents the majority of the potential log data, because by definition, the total size of the active transactions will be less than the total amount of RAM on a given computer. Thus, when circular logging is on, the system still has complete recoverability with respect to hard and soft failures. The element that is sacrificed is the extra protection against media failure. Also, because the transaction logs are used for incremental and differential backups, these backup methods are no longer valid. If you try to do an incremental or a differential backup on a server that has circular logging enabled, Microsoft Windows NT Backup will generate an error informing the user that it cannot be done.

When circular logging enabled you can see more than one transaction log in the log directory. For example, if a user sends a message with a 25 MB attachment it is likely, because the maximum log file size is 5 MB, to see four logs created to process this transaction in addition to Edb.log. That is, you will see Edb00001.log, Edb00002.log, Edb00003.log, and Edb00004.log. After the transaction is processed completely, these log files will remain until a full backup is performed on the database, at which time they will be removed. If the number of log files generated with circular logging enabled is equal to or greater than four (not including Edb.log) the logs will stay until a full backup is performed.

When circular logging is enabled, high volume conditions also cause the number of log files to increase. Because the inactive portion of a transaction log is only discarded and reused after the Edb.chk has moved past it, many transactions may build up in the logs before they can be processed. If the total number of unprocessed transactions exceeds 5 MB and more continue to build up, another log file (for example, Edb00001.log) will be created to hold these transactions. If the high volume conditions continue, the log files will continue to grow in number until the transactions can be processed completely. After the transaction processing catches up and the Edb.chk is advanced past the transactions in these log files, they will become inactive and again will not be removed until a full backup is performed. Other possible causes of log files growing with circular logging enabled are faulty SCSI controllers or hard drives that cause the transactions not to be completed in a timely manner and the Edb.log files being held open exclusively by another process other then Jet (for example, backup programs, copying operations, and so forth).