I had a client call me to day with a catastrophic failure of their Cyrus IMAP server running on Panther. They were getting this error in their Cyrus startup log:

DBERROR db4: 
    fatal region error detected; run recovery
DBERROR: dbenv->open '/var/imap/db' failed:  
DB_RUNRECOVERY: 
    Fatal error, run database recovery
    Fatal error: can't initialize mboxlist 
    environment
DBERROR db4: 
    fatal region error detected; run recovery
DBERROR: dbenv->open '/var/imap/db' failed: 
DB_RUNRECOVERY: 
    Fatal error, run Fatal error: can't 
    initialize mboxlist environment

Trying to reconstruct resulted in this error:

reconstruct:
    can't initialize mboxlist environment

Some searching in the Apple discussion forums led to this thread: 10.3.2 Mail Server: error on RECONSTRUCT which contains a fix by Dr. Fishopolis.

Since the Apple discussion forums are so hopelessly pokey right now I’ve decided to document the fix here (so I can find it again if I need it 🙂

Before you start: These instructions assume that the Cyrus mail store is in its default location: /var/spool/imap, and hasn’t been moved somewhere else (like to the XRAID for instance). Check the location in: Server Admin -> Mail -> Settings -> Advanced -> Mail store and replace any mention of /var/spool/imap with your sepecific path.

• Become root for a while:
  

  sudo -s

• Backup the /var/imap/ and var/spool/imap directories:
  

  ditto /var/imap/ /var/imap.bak/
  ditto /var/spool/imap/ /var/imap.spool.bak/

• Blow away the /var/imap/ directory:
  

  rm -r /var/imap

• And recreate it:
  

  /usr/bin/cyrus/tools/mkimap

• Correct the permissions:
  

  chown -R cyrus /var/imap
  chgrp -R mail /var/imap

• If the cyrus user doesn’t have a shell specified, do it now:
  

  niutil -createprop . /users/cyrus shell /bin/bash

• Become the cyrus user:
  

  su cyrus

• Now run the reconstruct command:
  

  /usr/bin/cyrus/bin/reconstruct

• You should see cyrus work through your mail accounts like this:
  

  user/user1
  user/user1/Folder1
  user/user1/Folder2
  user/user2
  user/user2/Folder1
  user/user2/Folder2

• If any of your users are missing personal mail folders reconstruct them individually using the account reference above (ie: user/user1) – You may need to run this command multiple times if the user has nested personal folders. Just run it until it stops producing errors:
  

  /usr/bin/cyrus/bin/reconstruct -r -f user/username

• Finally, stop being cyrus, then stop being root:
  

  exit
  exit

There, now wasn’t that fun.

Follow-up 1: There is associated problem pertaining to log files which is documented here.

Follow-up 2: Apparently there is an undocumented flag for the reconstruct command in the Panther version of Cyrus: "-i". It is supposed to rebuild nested user folders. I haven’t tested it (because I don’t have a broken email server right now :-)) but it is being recommended by Apple support.

Follow-up 3: Apple has posted a knowledge base article on the subject, read about it here.

Follow-up 4: Could it be we’ve discovered the ultimate fix for this? Read on.

(kinda like a snow day, only… not)

So, I’ve installed OS X 10.3.3, and WebObjects 5.2.3, and every thing is just working swimingly…

Happy, happy, happy, happy, I’m so happy