Zimbra upgrade2
27 Min Read
0
https://www.zimbra.com/docs/ne/4.5.10/administration_guide/10_Backup_Restore.13.1.html
Backing up the Zimbra server on a regular basis can help you quickly restore your mail service, if an unexpected crash occurs. The backup process writes a consistent snapshot of mailboxes to a designated backup directory.
• | Incremental backup files that contains the LDAP directory server files and all the redo log transactions written since the last backup |
• | Redo logs that contains current and archived transactions processed by the Zimbra server since the last incremental backup |
Figure 9 shows the sequence of a full point-in-time recovery. When a system is restored, the last full backup is restored, each incremental backup since the last backup is restored, and the archived and current redo logs are restored.
Figure 9: Sample backup timeline
Backup is performed using CLI commands. This chapter describes how Zimbra data is backed up and restored, and how to use the CLI tools to backup or restore your Zimbra server. In addition, this chapter also provides information and general guidelines for disaster recovery.
A full backup process backs up all the information needed to restore mailboxes, including the LDAP directory server, database, index directory, and message directory for each mailbox.
When backing up shared messages, the backup process looks to see whether a Binary Large Object file (BLOB) representing a message already exists in the backup. If it does, it simply flags this object as such and does not copy its content again.
An incremental backup process backs up the LDAP directory server and gathers all the redo log transactions written since the last incremental backup or the last full backup, if this is the first incremental since the last full backup. If the incremental backup process finds no previous full backup for a mailbox, a full backup is performed on that mailbox.
Incremental backups move the redo logs to the backup directory. The redo logs are a journal of every activity that has been logged. They contain a full copy of all BLOBs delivered, as well as metadata such as tags, contacts, and conversations.
These backup files can be used to restore the complete Zimbra system or individual mailboxes so that account and message data is completely restored.
The LDAP directory is backed up as part of either the full or incremental backup process. All accounts, domains, servers, COS, and other data are backed up. You can restore the LDAP directory without restoring the message server, and restore specific accounts to the LDAP server.
Each Zimbra server generates redo logs that contain every transaction processed by that server. If an unexpected shutdown occurs to the server, the redo logs are used for the following:
• | To ensure that no uncommitted transactions remain, the server rereads the redo logs upon startup. |
If the server is restored, after the backed up files are fully restored, any redo logs in the archive and the current redo log in use are replayed to bring the system to the point before the failure.
• | The Zimbra MTA is not backed up, as the data is only on the server for a very short time. |
The backup destination is known as a backup target. To the backup system, it is a path in the file system of the mail server. The Zimbra default backup directory is /opt/zimbra/backup.
The backup directory structure created by the backup process is shown in Figure 10. You can run regularly scheduled backups to the same target area without overwriting previous backup sessions.
Caution: Only create one backup target directory per Zimbra server. ZCS backup and restore does not support having more than one backup target.
Note: Keeping the same backup target saves disk space, because shared binary large object files (BLOB) and other files do not have to be duplicated every time the backup process runs.
Figure 10: Backup directory structure
The Zimbra backup process writes a consistent snapshot of mailboxes to the designated backup directory. Files are not overwritten. The default directory for the Zimbra backups is on the same mail server as is being backed up. This backup directory would be able to be used to restore one or more, or all user accounts. A disk drive can be mounted at the path to the backup target to store backup data, which can be transferred to tapes later.
Many of the backup and restore procedures can be run directly from the administration console. In the Navigation pane, Monitoring>Backup lists each of the servers. Select the server to perform the following backup and restore tasks.
The backup schedule is set by default during install, you can change this schedule from the command line. See „Scheduling Backups” .
The Zimbra backup and restore procedures are run as CLI commands. The following utilities are provided to create backup schedules, perform full and incremental backups, restore the mail server, or restore the LDAP server.
• | zmschedulebackup. This command is used to schedule full backups and incremental backups and add the backup schedule to your cron table. |
• | zmbackup. This command executes full or incremental backup of the mail server. This is run on a live server, i.e., while the Tomcat process and the mailbox server are running. In addition to full and incremental backup of the server, the zmbackup command can be used to backup specific accounts for archiving purposes. This command also has an option to delete old backups when they are no longer needed. |
• | zmbackupabort. This command stops a full backup that is in process. |
• | zmbackupabort -r. This command stops an ongoing restore. |
• | zmbackupquery. This command lists the information about ongoing and completed backups, including labels and dates. |
• | zmrestore. This command executes a full or incremental restore to the Zimbra mail server. The zmrestore command is performed on a server that is running. |
• | zmrestoreoffline. This command restores the Zimbra mail server when the Tomcat process is stopped. |
• | zmrestoreldap. This command restores the complete LDAP directory server, including accounts, domains, servers, COS and other data. |
When Zimbra Collaboration Suite is installed, by default, a backup schedule is automatically added to the Crontab. The default full backup is scheduled for 1:00 a.m., every Saturday. The default incremental backups are scheduled for 1:00 a.m., Sunday through Friday. This is the recommended schedule to have incremental backups run daily and a full backup run weekly. By default, backups older then a month are deleted on the first of each month at 12 a.m.
You can schedule backups for anytime or you can enter the command zmschedulebackup -D for the default schedule. The zmscheudlebackup command allows you to do the following:
• | Save the schedule command to a text file. This would allow you to easily recreate the same schedule after reinstall or upgrade. |
Backups are run using the Zimbra CLI command, zmbackup. This command performs full backups and incremental backups for all the accounts on a designated mailbox server. By default, the backup files are saved to the server’s backup directory. You can delete the backup files too.
When you initiate a backup, you can issue the command from the same server being backup up, or you can run the command remotely and specify the target server on the command line.
The full backup process goes through the following steps to backup the message store, the database, the indexes, and the LDAP directory:
2. |
3. | Iterates through each account to be backed up and backs up the LDAP entries for those accounts. |
4. | Places the account’s mailbox in maintenance mode to temporarily block mail delivery and user access to that mailbox. |
Full backup is usually run asynchronously. When you begin the full backup, the label of the ongoing backup process is immediately displayed. The backup continues in the background. You can use the zmbackupquery command to check the status of the running backup at any time.
Incremental backups are run using the CLI command, zmbackup. The process for incremental backup is as follows:
3. | Iterates through each account to be backed up and backs up the LDAP entries for those accounts. |
5. | Moves the archive redo logs, created since the last backup, to the |
If no full backup for this account is found, the backup process performs a full backup on this account, even if only an incremental backup was specified.
• | -t is not needed if the target is the default |
• | -zip can be added to the command line to zip the message blobs during backup. Zipping these can save backup storage space. |
• | Perform a full backup of all mailboxes on server1 to target at /mnt/disk. |
• | Perform incremental backup of all mailboxes on server1 since last full backup. |
• |
• |
Delete backup sessions either by label or by date. Deleting by label deletes that session and all backup sessions before that session. Deleting by date deletes all backup session prior to the specified date.
zmbackup -del 7d deletes backups older than 7 days from now. You can specify day (d), month (m), or year (y).
Each run of full or incremental backup creates a backup session, also known as the backup set.
The zmbackupquery command is used to find full backup sets. The command can be used to find a specific full backup set, full backup sets since a specific date, or all backup sets in the backup directory. Each backup session is automatically labeled by date and time. For example, the label backup200507121559510 says this is a backup from July 12, 2005 at 3:59.
Note: If a backup session is interrupted because the server crashes during backup (not aborted), the backup session that was interrupted was saved as a temporary session file. The backup session temporary file can be found at /sessions_tmp . You can use the rm command to delete the file.
You can use the CLI command, zmbackupabort to stop a backup that is in progress. The backup is immediately stopped and the files are marked as a failed session.
But before you can abort a backup, you must know the backup session label. This label is displayed when zmbackup first starts. If you do not know the full backup label, use zmbackupquery to find the label name.
• | The zmrestore command is used to restore the mail service while the Zimbra mailbox server is running. |
• | The zmrestoreoffline is used to restore the mail server when the mail server is down. This command is run for disaster recovery. |
• | The zmrestoreldap is used to restore the content of the LDAP directory server. |
The speed of the full versus incremental backup is inversely proportional to the ease of restoring. Full backups take longer to run, but the restore from a full backup is much faster. Incremental backups are quicker than full backups, but restoring from incremental backups is slower, because it re-applies the transactions in the redo logs.
In addition, you can use the zmrestore to restore LDAP entries for an account, restore an account to a new target account, and restore system and local config files.
The zmrestore process goes through the following steps to restore the message store, the database, the indexes, and the LDAP directory.
1. | Retrieves specified accounts of mailboxes to be restored, or specify all for all accounts that have been backed up. |
b. | Restores the last full backup of the MySQL entries, the index directory, and the message directory for that mailbox |
c. | Restores all incremental backups for that mailbox in order, since last full backup. This includes replaying the redo logs from the backup target area |
d. | Replays all archived redo logs for that mailbox, from the redo log archive area on the mailbox server |
Note: Before you run zmrestore, the accounts to be restored must be in maintenance mode. After the restore is complete, the accounts must be returned to active mode. This can be done from the administration console or from the CLI command, zmprov ma.
Important: After you have restored accounts, you should immediately run a full backup.
Important: Users using the ZCS Connector for Outlook must perform an initial sync on the Outlook client when they log on after the Zimbra server is restored.
• | Perform a full restore of all accounts on server1, including last full backup and any incremental backups since last full backup. |
• | Perform a single account restore on server1. |
• | Restore to a specific point in time (PIT). When you perform an incremental restore, you can specify a specific point in time to restore. The following restore options affect redo log replay. If you do not specify one of these options, all redo logs since the full backup you’re restoring from are replayed. |
IMPORTANT: After you perform any of the following point-in-time restores, you should immediately run a complete backup for those accounts to avoid future restoration problems with those accounts.
• | -restoreToTime |
• | -restoreToIncreLabel |
• | -restoreToRedoSeq |
• | -br – Replays the redo logs in backup only, which excludes archived and current redo logs of the system. |
• | -rf – Restores to the full backup only, does not include any incremental backups since that backup |
• | You can specify an exact time, the incremental backup label, or the redo log sequence to restore to. Restore stops at the earliest possible point in time if more than one point in time restore options are specified. |
• | Perform an incremental restore only to last full backup, excluding incremental backups since then, for all accounts on server1. |
The result from the above example would be an account called restoreaccount@company.com.
• | Include –contineOnError (-c) to the command so that the restore process continues if an error is encountered. Previously deleted accounts are not restored. If previously deleted accounts still exist on the server, they are deleted. |
When -c is designated, accounts that could not be restored are displayed when the restore process is complete.
Note: The restoration of a deleted account is only possible if the account existed at the last full backup and did not exist at a subsequent incremental backup. In order to restore an account if it was deleted as of the last full backup, you will need to find the last full backup that included the account. Specify the backup label (–lb) to restore the account.
The zmbackupabort -r command interrupts a restore that is in process. The restore process stops after the current account finishes being restored. The command displays a message showing which accounts were not restored before the interruption takes place.
The Zimbra offline restore process only can be run when the Tomcat server is not running. This tool is run for disaster recovery because the normal server startup sequence re-runs the latest redo log, but in a disaster recovery scenario, you would not want to do this until after performing a full restore, followed by an incremental restore. In general, offline restore is run under the following circumstances:
• | Certain components of the Zimbra server are corrupted, and the server cannot be started. For example, the data in LDAP or the database are corrupted. |
In a disaster recovery when the Zimbra software is reinstalled, if Tomcat is started before the backup files are restored, the mail server would begin to accept email messages and performing other activities, producing redo logs in the process. Since the pre-disaster data have not been restored to the server, the redo logs would be out of sequence. Once Tomcat is running, it would be too late to restore the pre-disaster data. For this reason, the offline restore must be run before the Zimbra server is started.
1. | Retrieves specified accounts of mailboxes to be restored. If the command-line does not specify any mailbox address, the list of all mailboxes on the specified mail host are retrieved from Zimbra LDAP directory server. |
c. | Restores all incremental backups for that mailbox in order, since the last full backup. This involves replaying the redo logs from the backup target area |
You must start Tomcat after the offline restore is complete. From the command line, type zmcontrol startup to start Tomcat.
In a disaster recovery where you need to restore not just one server, but the entire system including all devices, you should restore your LDAP directory server first.
The zmrestoreldap command restores the global LDAP data including COS, distribution lists, etc. You can restore the complete LDAP server, which recreates the entire schema or you can restore specific accounts. You specify the session to restore. The restore command has to be run on the LDAP server being restored.
The sequence of events to restore your Zimbra server in a general disaster scenario involving multiple machines would be as follows:
1. | Restore your LDAP directory server to a known good state before doing anything with the Zimbra server. |
2. | Put all Zimbra mailboxes into maintenance mode, to prevent mail delivery and user login while restoring the mailboxes. |
When you system is unexpectedly stopped and then restarted, on startup, the server automatically searches the redo log for any uncommitted transactions, and replays any that it finds. Replaying the redo logs brings the system to a consistent state in the event that the server was stopped due to a power loss or other event.
Important: The ZCS release you install on the new server must be the same release as installed on the old server. The server can have a different operating system.
The new server hardware must meet the requirements described in the Installation Prerequisites section of the ZCS Single Server Installation Guide. Install the new operating system, making any necessary OS configuration modifications as described in the installation guide.
Two scenarios for disaster recovery are the server has died and the ZCS files cannot be accessed, or ZCS is still running, but the server needs to be replaced.
2. | Run a full backup of the old service, or if the backup is recent, run an incremental backup to get the most current incremental backup session. |
3. | Run zmcontrol stop, to stop ZCS. In order to restore to the most current state, no new mail should be received after the last incremental backup has run. |
4. | Change the hostname and IP address on the old server to something else. Do not turn off the server. |
Before you begin, make sure that the new server is correctly configured with the IP address and hostname and that ZCS is installed and configured with the same domain, hostname, passwords, etc. as the previous server. See the Single-Server Installation Guide for more information about preparing the server. Before you begin to install ZCS, note the information you need from the old server including: admin account name and password, spam training and non-spam training user account names, exact domain name, and the global document account name.
1. | Copy your ZCSLicense.xml file to a directory on the new server. You will not be able to complete the ZCS installation if the license is not on the new server. |
2. | Follow the directions in the installation guide to install ZCS. Make sure that you configure the same domain, hostname, passwords as on the old server. During ZCS install, the following settings must be changed to match the original server settings: |
a. | Zimbra LDAP Server – For Domain to create – identify the same default domain as on the original server. |
b. | Zimbra Mailbox Server – An administrator’s account is automatically created. |
• | Make sure that the account name for Admin user to create is the same name as on the original server. |
• | Change the Spam training user and the Non-spam (HAM) training user account names to be the same as the spam account names on the original server. |
• | Global Document Account – This account name is automatically generated. Change the Global Document Account name to be the same account name as on the original server. |
c. | Change any other settings on the new server to match the configuration on the original server. |
1. | Stop the new server, type zmcontrol stop. |
2. | If the old server had additional storage volumes configured, mount any additional volumes now. |
3. | Delete the mysql data and re initialize an empty data directory. If you do not do this, zmrestoreoffline will have errors. As zimbra, type |
4. |
5. | To restore the LDAP, type zmrestoreldap -lb If you are restoring large number of accounts, you may want to run this command with nohup so that the session does not terminate. |
6. | Type zmconvertctl start. This is required before running zmrestoreoffline. |
7. | To start the offline restore, type zmrestoreoffline -sys -a all -c -br. You may want to run nohup here also. To watch the progress, tail /opt/zimbra/log/mailbox.log. |
Note: Use –c on the command line so that accounts will be restored even if some accounts encounter errors during the offline restore process.
8. | Because some ZCS services are running at this point, type zmcontrol stop to stop all services. |
9. | Remove any old backup sessions because these sessions are no longer valid. Type rm -rf /opt/zimbra/redolog/* /opt/zimbra/backup/* |
10. | To start ZCS, type zmcontrol start. |
11. | Now run a full backup, type zmbackup -f -a all. |
Use the zmrestore command to restore one or more selected accounts. In the event that a user’s mailbox has become corrupted, you might want to restore that user from the last full and incremental backup sets.
Note: You can also restore one account at a time from Accounts on the administration console.
1. | To put all accounts into maintenance mode, from the command line, enter zmprov ma |
The maintenance mode prevents delivery of new emails during the restore. Otherwise, the emails would be overwritten during the restore process.
2. | Run the zmrestore command to restore the accounts. Use commas between accounts. |
3. | Put all accounts into active mode, from the command line, enter zmprov ma zimbraAccountStatus active. |
Important: If an user account is restored and the COS that the account was assigned no longer exists, the default COS is assigned to the account.
The restoration steps are similar for most server failures you may encounter. If a failure occurs, review the disaster recovery section to understand the process and then follow the steps below for the specific type of failure.
2. | Find the label for the LDAP session to restore. Run the zmrestoreldap -lb command, with no arguments to restore all accounts, domains, servers, COS, etc. for the LDAP server. |
3. | Make sure that all accounts are in active mode. From the command line, enter zmprov ma zimbraAccountStatus active |
If any partition should become corrupted, (for example, the message store partition or the index partition), run zmrestore, to restore the latest full and incremental backup files. The zmrestore process automatically retrieves the list of all mailboxes on the specified mail host from the backup date and iterates through each mailbox to restore the mailboxes to the last known good state.
If the redo log becomes unreadable for any reason, the Tomcat service stops and cannot restart. If this happens, inspect the hardware and software to find the source of the problem before proceeding.
Without the latest redo log, the Zimbra mailbox server cannot be returned to the most a current state. The Zimbra mailbox data can be restored to the latest archived redo log state. A new redo log for current transactions is created after the Zimbra mailbox server is restored.
The Tomcat service must not be running and all accounts must be in maintenance mode before beginning.
1. | To put all accounts into maintenance mode, from the command line, enter zmprov ma zimbraAccountStatus maintenance |
2. | With the Tomcat service not running, type zmrestoreoffline. |
The offline restore process begins by retrieving the list of all mailboxes on the specified mail host from the backup .
• | Restore all incremental backups for that mailbox in order, since the last full backup. This involves replaying the redo logs from the backup target area |
Since the redo log for current transactions is not available, the mailbox server is returned to the state of the last archived redo log.
3. | Start Tomcat, after the offline restore is complete. From the command line, type zmcontrol startup. |
4. | Once the Zimbra mailbox server is up, run a full backup of the Zimbra server. The full backup must be run immediately to have the latest data backed up, as the latest redo log is not available. |
The localconfig.xml file, located in the /opt/zimbra/conf directory, includes the core Zimbra server configuration, such as paths and passwords, This file is backed up in full and incremental backups. When you run an incremental or full restore, the backed-up version of the localconfig.xml is renamed localconfig.xml.restore and is copied to the /opt/zimbra/conf directory.
If you have made changes since the last backup, you may need to replace the localconfig.xml file with the restored copy. Compare these files, and if the .restore file has the latest local configuration data, delete the localconfig.xml file and rename the file with the .restore extension to localconfig.xml.