Been using pmail for ~25 years and it's still my client of choice.

Currrently: Windows 7, Peg 4.72.572
(Yes, old, have been waiting for my new W11 laptop to arrive, which it finally has.)

Ran into a nasty problem recently: our sysadmins had to do a major renovation of our server and mentioned that email cache files would disappear and have to be recreated. With ~30K msgs it took a little while to read from the server, but was successful.

Except: I have color coded the headers of many messages, as a long-term to-do list. And that color coding is gone. This is a real problem for me.

Having encountered something like this in the past, I have a cron job that makes a backup of the Peg cache every day. Previously when there was a problem, I could go to
PMAIL/MAIL/ADMIN/IMC-3E9
and put a backup copy of the .PHC file there. Peg would read that, all the colors would be there, and I could pick up from there.

But now Peg insists on reading from the server, ignoring anything I put in IMC-3E9.

Help!

Any ideas about how to get Peg to read the PHC file on my machine first? (I see there are other folders like IMC -3E9, is there a way to use one of those, or a way to tell Peg I'm someone else, or ...??)

Many (ie ~30K) thanks!
Randy

It sounds like you are using a locally installed Pegasus Mail to access a server based mailbox via IMAP. The long hexadecimal name of a .PHC file associates it with a specific IMAP mail folder. If you had to created a new IMAP mail folder, well, that would explain why restoring a backup isn't working. The question then is whether you can identify the .PHC file currently being used and whether renaming the backup to match its name would work.
FWIW, I don't use IMAP so am just throwing out an idea.
Credit goes to Han van den Bogaerde for documenting the .PHC filename association in his guide to filenames and file-extensions.

You are talking about IMAP access, correct? Can you get screenshots of this IMAP profile? Feel free to obfuscate IMAP Server address and Login name.

Although I am an IMAP user, I'm not an expert. Analyzing the .PHC files, the headers downloaded from the server, I thought it would be reasonable to assume that they contain some key that links them to the remote messages. I could not find information about their architecture and I believe that attributes such as classification, grouping, colors, etc. of the messages should be handled locally in the PHC files or outside of them.

Playing around with some of my IMAP profiles, I was able to verify that two files outside of the IMAP directory were modified when I changed its sorting order: folders.pmj and IMAP.PM, the latter being the file for the IMAP profiles.

Pmail Help does not explain these particularities. It does offer an email address if the help there is not enough to address your problem. One thing you said rang a bell here, the "recreation of the cache" which means different PHC files. Maybe your colors still pointing to the old ones but I don't have the slightest idea where they are and how they are coded.

Sorry, and good luck!

To Brian:
Yes, locally installed PM using IMAP to server.
Yes, I know about the hex name of the PHC file and have in the past successfully replaced a newer PHC file (that is missing header colors) with one of the identical name, but retrieved from my backup collection, in particular far back enough that it had the colors.

To euler:
Yes, imap. I have included a pdf with screenshots of my IMAP profile data. As per my reply to Brian, I carefully name the backups to match the name of the .PHC files that Pegasus writes on my machine, so that isn't the issue.

Thanks in advance for any other suggestions/ideas.

recovering.pdf

In the next image, the current message color is back to RED. The color byte is back to 01h (the BLUE message is back to RED), but something new has changed and is marked in PURPLE. It is a single byte and probably a bitwise read. While working on the message, I accidentally opened it, which changed this byte. Resetting the message status to unread did not change this byte. The folder in question was also set to sort unread messages first, so opening the message changed the order of the messages. I imagine that this byte (and probably others) performs the functions of the PMFLAGS header of local messages.

The final image is the return to message's original color (BLACK) and unread status EXCEPT for the "PMFLAGS" byte.

What I can deduce from this superficial digging is that parameters such as color, order, grouping, etc. are saved in local PHC files. Without a detailed understanding of the structure of these files, I cannot go beyond that. It is quite likely that the fact that Pegasus ignores your backup and insists on getting new headers from the server has to do with some inconsistency between them. I think you should contact the Support as suggested. The email is in the Help > IMAP, Troubleshooting at the bottom.

I doubt you're going to like my response, but here it is. sigh.

Pegasus Mail's IMAP cache files are inherently transient - they are essentially just a bespoke database containing
copies of downloaded messages that allow the program to avoid unnecessary server traffic. The server can invalidate
any cache file at any time for any reason, and when this happens, Pegasus Mail clears the cache and starts building
it anew. My guess is that this is what has happened here - the server has invalidated the cache, which means
WinPMail will have deleted the old database and created a new one with a new ID (which is presumably why
the file you've "put back" is being ignored).

I can honestly say that it had never even occurred to me that someone might actually use a cache file as if it were
a real folder - in this case to take advantage of a client feature (message colouring) that the server doesn't
support. The code is simply not designed to support this, and the fact that it has apparently worked for you
in this way for some time is purely accidental. The "correct" way of handling this (by which I mean the way that
will work without the risk of data loss) is to move the messages from the IMAP message store into a local
folder and add your colouring there. My Mercury mail server has extended capability that allows Pegasus Mail
to set and store colour information on IMAP folders, but moving servers is unlikely to be a viable option for
you, I suspect.

I hate being the bearer of bad news here, but what you're trying to do is at odds with the way the program
is intended to work. I'm really sorry this has happened to you, but I doubt there's any worthwhile fix except
starting afresh using a local folder.

-- David --

Thanks so much for being willing to try to help out. One last gasp to see if I've got plan B that helps a little...

When our sysadmins installed the updates, they left a backup of the old server on another machine. One of them has suggested I might point my system toward that machine and then at least get a complete list of the colored msg headers as they were just before the upgrade.

I'd still have to go thru and mark the msgs on the new server (or as you say copy them to a their own folder), but this would at least get me a complete list.

Do that sound plausible?

Also: for future reference, how are header colors stored in the PHC file? (This being MIT we some good hackers here and with a little guidance I suspect one of them can write a routine to modify the new PHC file so I don't have to go thru and mark them by hand.)

More generally, is there documentation for the PHC file format? Instead of modifying the file it may be easier to enlist someone here to write a parser to simply produce an ascii listing with the relevant info for each msg (from, to, date, subject, color). I understand that the cache file is not a real folder; this is just gives a little useful info about the msgs it contains.

Thanks for any guidance you can offer. And of course thanks for an email client that is not designed to data mine its users.

Randy
MIT EECS

The backup on the other machine probably won't help you, because once again, the folder
UIDs will be different, even if the content of the folder is the same, meaning that WinPMail
won't be able to match the existing cache file to the new system. As I said in my previous
message, the .PHC files are inherently ephemeral, and they are very tightly-tied to the UID
assigned by the server for the folder they are caching.

For clarification: the colour information is stored in the cache, not on the server, so as long
as you can't match the cache to a current version of the folder on the server that has the
same server-assigned UID, the colour information is effectively lost. I'm assuming that
the folder on the server containing the actual messages wasn't somehow lost during the
upgrade, so you should be able to get a new, complete, accurate set of headers for the
folder simply by opening that folder on the new server: the headers won't have your
colouring, but would at least give you a starting point for doing it again, by moving or
copying the messages to a local folder and re-applying the colour data.

As far as how the colours are stored in the .PHC file... Well, that information really isn't of
much use to you because the file itself is a paged database, and without routines that
allowed you to read the data from the database, you couldn't easily access the data it
contains. This is unlike the native (and ancient) Pegasus Mail .PMM/.PMI format, where the
.PMI index file is simply a flat file of a predictable structure (which is why it loads so quickly).
The paged database manager is one of my own developments, so there isn't a convenient
open-source product you could download to allow you to read the files, either.

As for documentation on the cache file format - no, there isn't any apart from the source
code itself, I'm afraid, and I'd be reluctant to provide it even if I had it - and because that
sounds a bit unhelpful, I should explain why: in the early days of Pegasus Mail I documented
the .PMM/.PMI format and made it public, only later realizing that doing that made it almost
impossible to change the format at a later stage, which is why the folder format is so very
much a frozen creature of its time. The only way I could get around that was to develop a
completely new folder format, and I decided early on that I would not document it in any
way, instead providing a basic import/export facility for it.

Finally, the tenor of your message suggests that you might be planning to repeat the
process and try adjusting newly-created cache files the way you previously did, but I
truly recommend in the strongest possible terms that you don't do that, because sooner
or later, the very nature of the way the files are used will put you in the same position
again.

Sorry I can't offer you more than this.

-- David --

OTOH, opaque formats are a nightmare. What, if your exporter fails? What, if you get killed in a car accident, or whatever? In such a case, i'd see a much greater harm done, than might ever accure, following the "open" way.
Years ago i managed to convince a company's CEO to force his main developer to document his file formats - as he realized my reasoning to be valid, that noone apart from this guy knew the interna. In the end, i could show the developer, that even he had forgotten some all of his own design desicions...
Keep going and be well!
Karl

David:

Nope, learned the lesson the hard way here.
Thanks again for your patient and careful responses to my questions.