I have been using Pegasus Mail since the 1990s. I occasionally have had the odd issue, but I am now suffering an issue which may lead me to have to consider an alternative.

The issue is a frequent loss of the Mail Directory Structure, whereby Filing Trays are deleted, and message folders and many messages are dumped into the root of the structure, and some messages appear to be lost.

I am running V4.8, not having become aware of 4.91 although I do check quite regularly.

Is there any advice on offer,
A. To best correct this situation, and
B. To recover missing messages?

In hope,

Kenneth A. Spencer

The Folder list tray structure is a virtual structure whose records are stored in a file named HIERARCH.PM. When HIERARCH.PM gets corrupted, a new one is created by polling each folder file in the mailbox directory. Unfortunately, the mailbox files do maintain a record of any parent Tray they might have been in. The recovery method is to restore HIERARCH.PM from a recent backup. This may not work if the backup is too old. Do the restoration with Pegasus Mail closed.

A corrupted HIERARCH.PM will not result in any missing messages. The resulting rebuild can result in duplicate named folders not being displayed in the Folder list. If that is not it, post back with more specifics about the missing messages.

Best practices to prevent issues with HIERARCH.PM corruption included:

• Exclude the mailbox directory from active scanning by an anti-virus/anti-malware product.
• Never run two instances of Pegasus Mail that access the same mailbox.
• Backup regularly
• Use unique folder names. I practice I followed and encouraged all of my users to follow was to prefix the name of every nested folder with something the identified its parent Tray (eg: Tray name: "Accounting", Subfolder names prefixed with "Acc-"). This way, if an event resulted in being unable to restore HIERARCH.PM, then the folders would be sorted in the Folder list in a way that would group them by the parent Tray they had been in. Also, it helps minimize duplicate folder names.

Note: An abend during shutdown or an abrupt shutdown due to a power outage are unavoidable events that might result in a corrupted HIERARCH.PM.

Thank you Brian, for replying to my email.

First, some anwers to your implied questions:

1. I do a complete backup each night at 2:30am. This includes some 150GB of my most frequently changing data. Other material is backed up monthly (several TB).Copies of the most recent three backups are retained.

2. On this occasion there was no sudden crash or disconnection. I always allow adequate time for LAN activity after a logout to quieten before turning off a workstation.

3. I have a backup power supply unit on my servers and the principal workstation.

4. I think that I have recovered quite a lot by copying in my sub-directory from the backup 3 days ago, after forwarding to myself most (possibly not quite all) activity that occurred in the 3 days since the backup I used. I have collected those forwardings as replacements for most of the new data.

There is still some data missing, but I can't seem to find that, although very detailed searching is tedious and so I am tempted to leave it at that.

Thanks for your help.

What kind of data is missing?

I'm curious about exactly what you did. I assume a backup of your mail directory that would capture HIERARCH.PM is not included in your 2:30am backup so it sounds like you are restoring from the last monthly backup. If you made a copy(backup) of the mailbox directory before the restore then a sort by date comparison should identify the files that are newer than what is in the backup. Data files are .PMM, .PMI, and .CNM. The .PMM/.PMI file pair make up a folder, .CNM are new mail messages.

Thanks again, Brian, and I'll try to answer the questions:

1. The nightly backup is of the entire pmail directory, so it includes all users and their entire user data, so HIERARCH.PMs are included.

2. I have on occasion been able to restore the directory structure by copying in the backed-up HIERARCHY.PM when it has been unexpectedly lost, but on this occasion that did not work. Reverting to the backup from three days earlier more-or-less worked, but did not quite restore all data. The most important data lost was a single mail folder containing perhaps 100+ messages, but it seems to have disappeared completely.

3. Your information identifying the file extensions and their significance is helpful, and I shall keep it so as to help should I suffer any more incidents, so thanks for that.

If I may, one more question: is there a most frequent cause of this kind of failure (i.e. loss of directory structure) that I should be aware of? I do know that sudden disconnection or shutdown is significant, but anything else would be useful - I always watch for LAN activity to subside after closing Pegasus before shutting down a machine. Do you find that those who keep the data files on a workstation suffer fewer loss incidents than those who use a network server for data storage?

I shall fairly soon update to the latest version of PMAIL.

Many thanks for your attention,

Kenneth

Kenneth A. Spencer

There are no known issues with Pegasus Mail deleting folder files but disappearance from the Folder list due to a duplicate folder name is something I have seen numerous times. A tool that can help with analyzing what might be going on is called PegFolders. It will examine the folder files in the directory from which it is run and produce a report showing details the include folder name and filename. Running this on the current mailbox and then on a backup might reveal something.
PegFolder is available in the Add-on for Pegasus Mail category on this site.

If you found that helpful, you may find Hans' Guide to Filenames and Extensions even more so. https://www.vandenbogaerde.net/pegasusmail/pf_pmfiles.html

The issue is associated with a failure in the process of updating the HIERARCH.PM file during Pegasus Mail shutdown. An abend during shutdown can do it as can interference from an anti-virus/anti-malware product. The later is why excluding the mailbox directories from active scanning is encouraged.