I had a client call me to day with a catastrophic failure of their Cyrus IMAP server running on Panther. They were getting this error in their Cyrus startup log:
DBERROR db4:
fatal region error detected; run recovery
DBERROR: dbenv->open '/var/imap/db' failed:
DB_RUNRECOVERY:
Fatal error, run database recovery
Fatal error: can't initialize mboxlist
environment
DBERROR db4:
fatal region error detected; run recovery
DBERROR: dbenv->open '/var/imap/db' failed:
DB_RUNRECOVERY:
Fatal error, run Fatal error: can't
initialize mboxlist environment
Trying to reconstruct resulted in this error:
reconstruct:
can't initialize mboxlist environment
Some searching in the Apple discussion forums led to this thread: 10.3.2 Mail Server: error on RECONSTRUCT which contains a fix by Dr. Fishopolis.
Since the Apple discussion forums are so hopelessly pokey right now I’ve decided to document the fix here (so I can find it again if I need it 🙂
Before you start: These instructions assume that the Cyrus mail store is in its default location: /var/spool/imap, and hasn’t been moved somewhere else (like to the XRAID for instance). Check the location in: Server Admin -> Mail -> Settings -> Advanced -> Mail store and replace any mention of /var/spool/imap with your sepecific path.
- Become root for a while:
sudo -s
- Backup the /var/imap/ and var/spool/imap directories:
ditto /var/imap/ /var/imap.bak/ ditto /var/spool/imap/ /var/imap.spool.bak/
- Blow away the /var/imap/ directory:
rm -r /var/imap
- And recreate it:
/usr/bin/cyrus/tools/mkimap
- Correct the permissions:
chown -R cyrus /var/imap chgrp -R mail /var/imap
- If the cyrus user doesn’t have a shell specified, do it now:
niutil -createprop . /users/cyrus shell /bin/bash
- Become the cyrus user:
su cyrus
- Now run the reconstruct command:
/usr/bin/cyrus/bin/reconstruct
- You should see cyrus work through your mail accounts like this:
user/user1 user/user1/Folder1 user/user1/Folder2 user/user2 user/user2/Folder1 user/user2/Folder2
- If any of your users are missing personal mail folders reconstruct them individually using the account reference above (ie: user/user1) – You may need to run this command multiple times if the user has nested personal folders. Just run it until it stops producing errors:
/usr/bin/cyrus/bin/reconstruct -r -f user/username
- Finally, stop being cyrus, then stop being root:
exit exit
There, now wasn’t that fun.
Follow-up 1: There is associated problem pertaining to log files which is documented here.
Follow-up 2: Apparently there is an undocumented flag for the reconstruct command in the Panther version of Cyrus: "-i". It is supposed to rebuild nested user folders. I haven’t tested it (because I don’t have a broken email server right now :-)) but it is being recommended by Apple support.
Follow-up 3: Apple has posted a knowledge base article on the subject, read about it here.
Follow-up 4: Could it be we’ve discovered the ultimate fix for this? Read on.