[quote user="CraigSpencer"]Looking over Bogaerde's  file list I found CACHE.PM which may explain the behavior I observed. It seems that the folders are loaded without inspection if the existing folders match the list maintained here. But if a folder has been deleted or a new one is found then an initial check will be made of all folders.[/quote]

I have no idea how the cache actually works, you may be right here.

Hello,

I had a computer problem that resulted in my having to get a new disk and begin reconstructing my XP system. I downloaded a new copy of Pegasus 4.52 and installed it.

Unfortunately, I do not have a perfect copy of my old Pegasus MAIL folder. :( But I'd like to recover as much as I can of it. I copied all the old contents into the MAIL folder of the new Pegasus and was delighted to see that almost everything appeared to be intact. There were a few corrupt items in the NewMail folder. However, all those items appear to be independent files which I could just delete (with Pegasus) will no apparent ill effects on the Pegasus system. 

On close inspection of the mailboxes some of them seem to have some problems altho most are intact. I discovered the "check consistency" feature which seems to allow me to test them for problems. This lead to trying the "reindex" feature on a few of the problem mailboxes I found. I was delighted that it seemed to fix them. At least it fixed the first few I tried.

However, I have found a few that it cannot fix. And a few that it claims to fix but doesn't really. OK, I didn't expect to recover everything and would be delighted if I could just save everything that is intact.. 

I thought I ought to be able to just delete these unfixable mailboxes. I thought each mailbox was an independent file with its own internal indexing independent of the rest of the system. So I thought that if I deleted (with Pegasus) the bad ones the rest of the system should continue to work.

However, I have found that this does not seem to be the case. I found one which, if deleted (with Pegasus) seems to screw up the entire mailbox tree. Can anyone tell me how is this possible? Any suggestions as to how to proceed would be most welcome.

It is frustrating that nearly all of what I want is intact but I can't seem to get rid of the bad parts without corrupting the good parts.

Best,

Craig

[quote user="CraigSpencer"]However, I have found that this does not seem to be the case. I found one which, if deleted (with Pegasus) seems to screw up the entire mailbox tree. Can anyone tell me how is this possible? Any suggestions as to how to proceed would be most welcome.[/quote]

Sounds like a "virtual" issue as your assumption about folder files and indexes is correct, most probably your issue is caused by a corrupted HIERARCH.PM file in your mailbox directory: If you right-click a folder and select Folder information you'll get its filename and location displayed, HIERARCH.PM is stored in the same directory (if the path is truncated got to Help => About Pegasus Mail => Info and pick Home mailbox location as the proper path).

You may simply delete HIERARCH.PM if you don't use lots of trays in your mailbox, otherwise you'd rather like to open it with a plain text editor such as the former NotePad and remove apparently corrupted lines. Lines containing Name_Unavailable may be removed as well, but close down Pegasus Mail before fiddling with its files.

Once you have the folder's name and location you may remove it (and its associated index file with PMI extension) as well, if required, but simply removing only the index file and reindexing the folder (from within Pegasus Mail) might help as well. Always keeping backups is a good idea, of course ...

Thanks for your reply.

I'm not sure what you mean by a "virtual" issue.

I did see in some of your other posts recommendations that the MAIL not be under Program Files. My PMail folder (which contains Pegasus' MAIL and PROGRAMS folders) is in Program Files. That is where I have always put it and it has always worked perfectly. Of course, I am using XP. Perhaps your warning only pertained to W7 etc.? 

I found HIERARCH.PM in my MAIL file. I have a rich hierarchy of trays and mailboxes so I don't want to lose that. If I were to delete HIERARCH.PM would I lose the mailboxes stored in the hierarchy? Or would all the mailboxes remain, but in a flat hierarchy? If the later, I suppose I could reconstruct my hierarchy as long as I didn't lose the contents. If I delete HIERARCH will Pegasus recreate it with a flat hierarchy for all the mailboxes it finds in MAIL? Or what?

I took an initial look at HIERARCH.PM with NotePad. I saw no obvious corruption. That is, there was no obvious gibberish. I did see a number of lines containing Name_Unavailable. Is that normal or an indication of corruption? Are those lines left over from normal deletion processes and the files they reference no longer exist? Or do they mean something is wrong with the files?

Do you mean to say I should remove mailboxes in a Name_Unavailable line?

I have been reluctant to remove mailbox files manually. I have only tried deleting them from within Pegasus. That produced the disastrous results I previously described (at least for a "bad" mailbox) so I am reluctant to just do it by hand. Since I didn't understand that result I didn't know if that would be safe.

Will removing the .PMI file and then clicking on the mailbox with reindex do anything different than just reindexing it otherwise?

I am hoping you can tell me more about how things work so I can figure out how to proceed.

Craig

[quote user="CraigSpencer"]I'm not sure what you mean by a "virtual" issue.[/quote]

A virtual issue is a display issue not based on real data resp. file corruption: Deleting folder files (+ index) cannot affect other folders unless the files are cross-linked due to severe system or harddisk failures which can be checked and possibly fixed by running chkdsk in a commandline window (chkdsk /? displays commandline options) - but fixing may also result in loss of data.

[quote user="CraigSpencer"]Perhaps your warning only pertained to W7 etc.?[/quote] Windows 7 (and most probably Vista) as referred to in these posts.

[quote user="CraigSpencer"]I am hoping you can tell me more about how things work so I can figure out how to proceed.[/quote]

Why don't you just try all the other stuff to figure out whether it helps? I can't really predict what's going to happen as I don't have an idea what the reason for your issue is, I would have to try the same: As long as you keep a backup of the whole MAIL directory resp. HIERARCH.PM and your folder files you won't need to be afraid of loosing anything. Working on HIERARCH.PM as suggested won't cause any loss of emails or folders, it will only affect the tray structure, and deleting an index file won't cause any loss of messages either, but some message flags may get lost (you probably know that this can happen due to reindexing).

BTW: We should use exact terminology here: Usually Pegasus Mail only displays a single mailbox containing trays and folders. There is a feature for adding additional mailboxes, but then again each mailbox will contains trays and folders, and if I understand correctly you're talking about folders, not about mailboxes?

[quote user="idw"]

[quote user="CraigSpencer"]I'm not sure what you mean by a "virtual" issue.[/quote]

A virtual issue is a display issue not based on real data resp. file corruption: Deleting folder files (+ index) cannot affect other folders unless the files are cross-linked due to severe system or harddisk failures which can be checked and possibly fixed by running chkdsk in a commandline window (chkdsk /? displays commandline options) - but fixing may also result in loss of data.

[/quote]

OK. So something is causing Pegusus to display nonsense unrelated to anything in the files. Perhaps the display is related to something wrong with the HIERARCH file caused by the deletion?

It is a new disk and a fresh installation so I don't think there is any chance of a file system problem. All there could be is bad data in the files as a result of file system problems on the previous disk.

[quote user="idw"]

[quote user="CraigSpencer"]I am hoping you can tell me more about how things work so I can figure out how to proceed.[/quote]

Why don't you just try all the other stuff to figure out whether it helps? [/quote]

OK, you have given me things to try. Thanks. I just was hoping to get a better understanding of a few things about how Pegasus is organized to guide me.

 eg. Was I correct in my speculation that deleting the HIERARCH file will cause it to be regenerated with a flat hierarchy including all the folders in the mailbox?

Also, if you could tell me more about the significance of the "Name Unavailable" lines I'd much appreciate it. Does that mean some sort of corruption or error or is it normal (eg the result of an ordinary deletion)? I really have no idea what is the meaning of these lines and I would be much obliged if you could spare me having to try to figure it out by experiment.

[quote user="idw"]

BTW: We should use exact terminology here: Usually Pegasus Mail only displays a single mailbox containing trays and folders. There is a feature for adding additional mailboxes, but then again each mailbox will contains trays and folders, and if I understand correctly you're talking about folders, not about mailboxes?

[/quote]

Quite right. I was talking about the folders containing saved messages. Sorry. I thought I was using Pegasus terminology correctly. Thanks for correcting me so that I can communicate about this better. 

Thanks a lot for your help!

If anyone can tell me what is the function of the .PM$ files in the MAIL folder, I'd appreciate it. They seem to contain data from old emails. Perhaps deleted ones. I am not sure. The ones I checked all seemed to be multipart or html rather than plain text. I'm not sure if that has anything to do with it.

I thought I'd record my observations here, just in case someone might come looking for such information again.

Looking at the HIERARCH file, none of the "Name Unavailable" lines refered to Pegasus folders that actually exist in the MAIL folder. So I would guess that these are normal remnants of deleted folders and not indicative of any corruption or error.

Deleting the HIERARCH file and then running Pegasus does cause it to be regenerated as a flat hierarchy.

On the other hand, deleting all the .PMM and .PMI files leaves the hierarchy intact and the tree still displays (minus the folders deleted). The references in the HIERARCH file all get changed to "Name Unavailable", however. Replacing a folder (a .PMM and .PMI pair) causes it to reappear in the tree and the name reference in the HIERARCH file is restored as well. 

I observed in the flat version containing some bad folders that deleting a good (well indexed) folder pair can cause the display (at least) of the folder/tray tree to produce garbage. This happens either by deleting the file pair directly or by deleting the folder from inside Pegasus. When this happens, the HIERARCH file acquires "Name Unavailable" labels for lines refering to files that do exist (perhaps bad ones) and there is garbage added to the end of the HIERARCH file. This garbage appears to be generated by Pegasus as it is in the form of "normal" lines with question marks or garbage strings replacing the folder names. Replacing the deleted files does not restore the previous state. It does, however, restore the correct line with name for the replaced files.  Any explanation for this behavior would be welcome.

Craig

[quote user="CraigSpencer"]I observed in the flat version containing some bad folders that deleting a good (well indexed) folder pair can cause the display (at least) of the folder/tray tree to produce garbage. This happens either by deleting the file pair directly or by deleting the folder from inside Pegasus. When this happens, the HIERARCH file acquires "Name Unavailable" labels for lines refering to files that do exist (perhaps bad ones) and there is garbage added to the end of the HIERARCH file. This garbage appears to be generated by Pegasus as it is in the form of "normal" lines with question marks or garbage strings replacing the folder names. Replacing the deleted files does not restore the previous state. It does, however, restore the correct line with name for the replaced files.  Any explanation for this behavior would be welcome.[/quote]

Try turning off your AV scanner for Pegasus Mail file types or your entire mailbox directory.

[quote user="CraigSpencer"]If anyone can tell me what is the function of the .PM$ files in the MAIL folder, I'd appreciate it.[/quote]

For details about Pegasus Mail's files go to Han v.d. Bogaerde's page.

[quote user="idw"]

Try turning off your AV scanner for Pegasus Mail file types or your entire mailbox directory.

[/quote]

I tried that but it didn't make any difference.

 Craig

[quote user="idw"]

[quote user="CraigSpencer"]If anyone can tell me what is the function of the .PM$ files in the MAIL folder, I'd appreciate it.[/quote]

For details about Pegasus Mail's files go to Han v.d. Bogaerde's page.

[/quote]

That is a great reference. Thanks.

Craig

I have to say that I am still at a loss to understand what I am observing.

There is no information in the HIERARCH file about the size or organization of the folders. If it is deleted Pegasus seems to have no problem to compile a new list of all the folder-pairs in the MAIL file (even tho some of them may be corrupt or malindexed) and list them in a flat mailbox. It is just a list of files irrespective of their contents except, apparently, the fetching of the name.  

Yet, deleting a pair (internally or externally) seems to cause an erroneous recompilation of the list due to some of the files being corrupt.

Craig

[quote user="CraigSpencer"]I have to say that I am still at a loss to understand what I am observing.[/quote]

Sometimes it's up to people like you to figure out what's going on if nobody else can duplicate it, and I'm afraid this is one of these cases ...

Writing this prompted me to do an experiment. Recall that when I originally deleted the HIERARCH file Pegasus recompiled an apparently complete list of the folders. Well, looking closely it seemed to get every single name correct, even tho a few of the folders are corrupt. So it was able to get correct information out of every single file (or, perhaps, the names came from some other source?). 

So for my experiment, I deleted a good folder. As before that produced a defective HIERARCH on next loading of Pegasus. Then I deleted the HIERARCH file to see if that would again produce a correctly recompiled list of the remaining folders. :( It didn't.

It seems to me that there must be some other source for information about the folders other than the folders themselves and the HIERARCH file! Something that notices if one of the original folders is missing and recompiles something as a result using internal information from one of the corrupt folders. This last does not seem to be done if the list of folders has not changed. How can Pegasus know if the folder list has changed without a HIERARCH file?

[quote user="CraigSpencer"]So for my experiment, I deleted a good folder. As before that produced a defective HIERARCH on next loading of Pegasus. Then I deleted the HIERARCH file to see if that would again produce a correctly recompiled list of the remaining folders. :( It didn't. [/quote]

Please be exact with the information you provide: How did you delete the folder (internally or externally)? Did you close Pegasus Mail before deleting HIERARCH.PM? If you didn't Pegasus Mail will most probably restore the current one on exit.

[quote user="CraigSpencer"]It seems to me that there must be some other source for information about the folders other than the folders themselves and the HIERARCH file![/quote]

No, there isn't. All the information used by Pegasus Mail is stored in the first 128 bytes of a PMM file (the folder name, e.g) and the index file.

Do you have multiple folders with identical names, BTW? Are you sure all instances of Pegasus Mail are closed while doing your experiments? Could it be a hidden instance still "hanging" in memory (use the Windows task manager (right-click the taskbar) for detecting and closing it after closing any visible instance the normal way, don't only check the Applications page, the Processes page might show another WINPM-32.EXE process even though the Applications page wouldn't do so)? What about keeping corrupted folders outside of your mailbox directory for further testing? How do you tell these folders are actually corrupted?

[quote user="idw"]

[quote user="CraigSpencer"]So for my experiment, I deleted a good folder. As before that produced a defective HIERARCH on next loading of Pegasus. Then I deleted the HIERARCH file to see if that would again produce a correctly recompiled list of the remaining folders. :( It didn't. [/quote]

Please be exact with the information you provide: How did you delete the folder (internally or externally)? Did you close Pegasus Mail before deleting HIERARCH.PM? If you didn't Pegasus Mail will most probably restore the current one on exit.

[/quote]

I deleted it internally. Then I closed Pegasus and deleted the HIERARCH file. Upon restarting Pegasus, the recreated HIERARCH file was erroneous. 

However, I just repeated the experiment: I deleted both the good folder/pair and the Hierarch file at the same time. When I started Pegasus again the result was the same.

I also did this (deleting the folder and the HIERARCH file) with the MAIL file from which I successfully created the flat version in the first place. The result was the same.

Yes, I closed Pegasus before fiddling with any of its folders.

[quote user="idw"]

[quote user="CraigSpencer"]It seems to me that there must be some other source for information about the folders other than the folders themselves and the HIERARCH file![/quote]

No, there isn't. All the information used by Pegasus Mail is stored in the first 128 bytes of a PMM file (the folder name, e.g) and the index file.

[/quote]

That makes it very hard to understand what I am observing. I'm sure there is a good explanation but I don't see it.

[quote user="idw"]

Do you have multiple folders with identical names, BTW?

[/quote]

I believe so. There were two folders names "Accounts" that were in different parts of the tree. The names of the .PMM and .PMI files are different and the name is only an alias so I could not see how that could matter. Does it?

[quote user="idw"]

Are you sure all instances of Pegasus Mail are closed while doing your experiments? Could it be a hidden instance still "hanging" in memory (use the Windows task manager (right-click the taskbar) for detecting and closing it after closing any visible instance the normal way, don't only check the Applications page, the Processes page might show another WINPM-32.EXE process even though the Applications page wouldn't do so)?

[/quote]

Yes, all instances of Pegasus are closed. I found no indication otherwise in the Task Manager. 

[quote user="idw"]

What about keeping corrupted folders outside of your mailbox directory for further testing?

[/quote]

If I can't remove files I can't separate them. Of course, maybe removing all the bad files will produce better results. I'll try that.

[quote user="idw"]

How do you tell these folders are actually corrupted?

[/quote]

Pegasus' consistency check.

If it fails the consistency check, I run reindex and afterwards it still fails the consistency check I figure it is terminally corrupted.

I also inspect the contents by opening them in Pegasus. I have to say many of these folders look OK until I reindex them. My guess would be that that is because the information comes out of the .PMI file which is OK until it is regenerated from the .PMM file which is corrupt.

Craig

[quote user="CraigSpencer"][quote user="idw"]Do you have multiple folders with identical names, BTW?[/quote]

I believe so. There were two folders names "Accounts" that were in different parts of the tree. The names of the .PMM and .PMI files are different and the name is only an alias so I could not see how that could matter. Does it?[/quote]

Yes, I don't know whether it affects your issue, but it can confuse Pegasus Mail in certain cases where it doesn't seem to use the unique IDs but the alias names (copy to self is such a case as far as I remember).

[quote user="CraigSpencer"]If I can't remove files I can't separate them.[/quote]

What does that mean I can't remove files? If you can't remove files than you have a serious problem, but not with Pegasus Mail ...

[quote user="CraigSpencer"]Pegasus' consistency check.

If it fails the consistency check, I run reindex and afterwards it still fails the consistency check I figure it is terminally corrupted.[/quote]

Did you try after deleting the associated index file(s)? You may also try mbxmaint_ui.exe from Pegasus Mail's program directory (after closing Pegasus Mail, of course).

[quote user="CraigSpencer"]I also inspect the contents by opening them in Pegasus. I have to say many of these folders look OK until I reindex them.[/quote]

That still doesn't give me a clue what a corrupted folder looks like in your case: What's wrong with it? Maybe you're hunting a phantom?

[quote user="idw"] [quote user="CraigSpencer"][quote user="idw"]Do you have multiple folders with identical names, BTW?[/quote]

I believe so. There were two folders names "Accounts" that were in different parts of the tree. The names of the .PMM and .PMI files are different and the name is only an alias so I could not see how that could matter. Does it?[/quote]

Yes, I don't know whether it affects your issue, but it can confuse Pegasus Mail in certain cases where it doesn't seem to use the unique IDs but the alias names (copy to self is such a case as far as I remember).[/quote]

Do you mean that "copy to self" is a special name which cannot be duplicated?

[quote user="idw"]

[quote user="CraigSpencer"]If I can't remove files I can't separate them. [/quote]

What does that mean I can't remove files? If you can't remove files than you have a serious problem, but not with Pegasus Mail ...

[/quote]

I mean that when I remove files then the behavior of Pegasus becomes unreliable so I no longer can use it to work on the folders.  

[quote user="idw"]

[quote user="CraigSpencer"]Pegasus' consistency check.

If it fails the consistency check, I run reindex and afterwards it still fails the consistency check I figure it is terminally corrupted.[/quote]

Did you try after deleting the associated index file(s)? You may also try mbxmaint_ui.exe from Pegasus Mail's program directory (after closing Pegasus Mail, of course).

[/quote]

Yes, I tried deleting the index file once. It didn't seem to make any difference.

What is mbxmaint_ui.exe? Some sort of utility program?

[quote user="idw"]

[quote user="CraigSpencer"]I also inspect the contents by opening them in Pegasus. I have to say many of these folders look OK until I reindex them.[/quote]

That still doesn't give me a clue what a corrupted folder looks like in your case: What's wrong with it? Maybe you're hunting a phantom?

[/quote]

An obviously corrupted folder, when opened in Pegasus, will have some sort of garbage displayed. Usually it will be in the form of a bunch of untitled, zero length "messages" with the same time and date as the previous good message.

Sometimes if you open a message it will contain (partial) gibberish.

I have also seen folders whose properties will be 300,000 bytes length and 500,000,000 deleted space.

Craig

[quote user="CraigSpencer"]Do you mean that "copy to self" is a special name which cannot be duplicated?[/quote]

No, I mean: If you set up Pegasus Mail to ask for a folder to put your copy to self in it may not put it into the folder you select if there are duplicate folder names. IOW: Don't do this (using folders with identical names, that is) as it might fail on other occasions as well.

[quote user="CraigSpencer"]What is mbxmaint_ui.exe? Some sort of utility program?[/quote]

Sure, it'll reformat your harddisk [;)]: Mailbox Maintenance with User Interface tool.

[quote user="CraigSpencer"]I mean that when I remove files then the behavior of Pegasus becomes unreliable so I no longer can use it to work on the folders.[/quote]

So there's only ony possible solution IMO: Set up a completely clean new version of Pegasus Mail, open each PMM file with NotePad or whatever else your plain text editor (no Office application!) is named, note the folder's name (right at the start of the file), move the file into the new clean home mailbox directory (see Help => About Pegasus Mail => Info for its location), restart Pegasus Mail, reindex the now unnamed folder and rename it to its previous name (or a new one taking care of the duplicate name issue, of course).

If such a folder still looks corrupted close Pegasus Mail, open the PMM file again and remove the first 128 bytes completely, then reindex and rename again. Please note that all the file contents after byte 128 should be plain text (not necessarily readable because of MIME encodings), if there's any garbage in there than the respective message can't be recovered at all (only the message delimiter (termination character) is a CTRL + Z char looking like an arrow to the right like this: ->).

If you got plenty of folders you may prefer to only apply the above to the corrupted folders.