Pegasus Mail & Mercury

Where can I find documentation on directly accessing/parsing a .PMM file? It seems to consist of messages prefixed with additional flags?

 

This information is not to be spread widely. Mail me directly (include reason why) and I can try to see what I have stored over the years.

Han.

---

Han van den Bogaerde - support@vandenbogaerde.net
Member of Pegasus Mail Support Group.
My own Pegasus Mail related web information:
http://www.vandenbogaerde.net/pegasusmail/

Han v.d. Bogaerde:

This information is not to be spread widely.

Han, thank you for this, but I think these days the requirement for secrecy is really a pretty laughable facet of the older, more arrogant David. The modern, less concerned David really doesn't mind if this information makes its way into the public domain - indeed, I think there's a real merit in having this type of thing in the FAQs section here.

It's quite possible that you actually have considerably better resources for this type of documentation than I do at the moment, so if you can put something together, please feel free to make it available. If you haven't, let me know and I'll see what I can dig out: I know I wrote some basic descriptions of the various formats a number of years ago, and I can probably still find them with a little effort.

Cheers!

-- David --

David, this is the last I've gotten from the old Admin manual but I know that there have been changes.


Folders:

A Pegasus Mail 2.2 folder consists of two files. The master file has the extension .PMM: it has a 128-byte header of which the first 50 bytes are the long name of the folder, the remainder being reserved. Following the header is the text of all the messages in the folder, separated by ^Z characters (ASCII 26). Some of the messages stored in the folder may in fact be deleted  there is no way of determining this without consulting the record matching the message in the .PMI index file for the folder.

The other file has the extension .PMI, and consists of a
representation of the message using a structure called an IMESSAGE, shown below in its C language definition (note that all integer values in the file are stored in Intel word order  Pegasus Mail for the Macintosh does whatever conversion is necessary for its Motorola processor as it loads each entry from the index):

typedef struct
{
unsigned long flags;
unsigned long fpos; /* Offset in master file */
WORD msg_number; /* Msg ordinal position */
char fname; /* Unused in folders */
char from [30]; /* Sender of message */
char subject [36]; /* Guess what this is */
char date [20]; /* Reduced form of date */
long mtime; /* See below */
long fsize; /* Bytes in this message */
} IMESSAGE;

flags is a bitmap of message characteristics, using the following values:

0x1 The message has Pegasus Mail-style attachments
0x2 The message is a uuencoded file transmission
0x4 The message is encrypted
0x80 The message has been read
0x2000 Sender requests confirmation of reading
0x20000L The message is a copy to self
0x40000L The message has been deleted
0x80000L The message is in a MIME transmission format
0x100000L A reply has been sent for this message
0x200000L The message has been forwarded to another user
0x400000L The message is urgent (never seen in folders)
0x800000L The message contains BinHex-encoded enclosures
0x1000000L The message originates from an MHS system
0x2000000L The message originates from an SMTP system.
0x4000000L The message has annotations.
0x8000000L The message contains enclosures

All other values are reserved - do not use them.

The mtime field is a crude calculation of the number of seconds since Jan 1 1990, used only for sorting purposes. It is not intended to be accurate merely "near enough".

The fsize field can be larger than the actual size of the message, but in no circumstance should it be less - this will cause Pegasus Mail to crash.

---

Thomas R. Stephenson
San Jose, California
Member of Pegasus Mail Support Team

Thanks guys, these are very interesting information. Just a few additional queries:

• What does the "mtime" field stands for? The equivalent of date field or something else?
• I don't suppose a programming API is available for people who want to use the "official" way to read/write PMM files as well as navigating the folder hierarchy rather than asking them to parse *.PMM and HIERARCH.PM files themselves?

 

 

pmuser:

Thanks guys, these are very interesting information. Just a few additional queries:

• What does the "mtime" field stands for? The equivalent of date field or something else?

"mtime" is a crude hash value based on the date which is used only when
sorting the folder. The code which calculates this value is as follows:

    long extract_time (char *s)
       {
       int year, mon, day, hour, min, i;
       char monthstr [10], *base;
       long l;

       /*  Extract a date in RFC-822 date format, and:
       **  i:   Sprintf() it into a consistent format for display.
       **  ii:  Calculate a sequence value. The sequence value is
       **       used for sorting by date, and is crudely calculated.
       **       It is very nominally the number of minutes since
       **       Jan 1 1990, but in fact it ends up being nothing like
       **       this. This doesn't matter provided the sequence it
       **       produces is correct (which it is).
       */

       if (s [3] == ',')
          base = s + 4;
       else
          base = s;

       if (sscanf (base, "%d %s %d %d:%d", &day, monthstr, &year,
       &hour, &min) != 5)
          return 0L;

       for (mon = 0, i = 0; i < 12; i ++)
          if (stricmp (monthstr, months ) == 0)
             {
             mon = i;
             break;
             }

       if (year > 1900) year -= 1900;
       l = (year - 90) * 535680L;          /* 12 * 31 * 1440 (months) */
       l += (mon * 44640L);                /* 31 * 1440 (days)        */
       l += (day * 1440L);                 /* 60 * 24 hours           */
       l += (hour * 60L);                  /* 60 minutes              */
       l += min;

       sprintf (s, "%2d %s %02d %2d:%02d", day, monthstr, year, hour, min);
       return l;
       }
 

 

---

Han van den Bogaerde - support@vandenbogaerde.net
Member of Pegasus Mail Support Group.
My own Pegasus Mail related web information:
http://www.vandenbogaerde.net/pegasusmail/

Thank you for posting this.  This is, however, exactly what is in a file called UNIX2PM.H which is part of an archive that purports to try to convert MBOX to Pegasus mail.  Your exposition above is wrong, also.  The fname field is a char [14], not a single char.

 Also, upon viewing a hexdump of a PMI file, there is a  4 character (probably unsigned long?) value at the end of the date field, making the length of the date field 16 instead of 20.

 David, could you please clarify?

 
Also, and I maybe should be posting this in another new thread, but I have the feeling that these fields are related, there is a problem with annotations.  The old style of annotations had an "A" as the first letter and were able to be easily found if one knew the name of the PMM file.  Now, the annotation filenames are 16 byte hash values which have no obvious connection to the message they pertain to.

The old annotations are lost.  This is a bug. If there is any documentation about what to do about this to convert the annotations to the new format, please show me.
 

Here is the latest V2 format for the PMI file:

 The index file:  the .PMI file contains a preparsed version of each
message in the file represented in a flat structure. The file is not
sorted in any way and has no header component. The structure used in the
index file is as follows:

typedef unsigned long ULONG
typedef unsigned short USHORT
typedef short INT_16
typedef unsigned char UCHAR

typedef struct
   {
   ULONG flags;          /* (*See details below)
   ULONG fpos;           /* Offset within 2.2 folder to this message */
   USHORT msg_number;    /* Ordinal position of this IMESSAGE in index */
   char fname [14];      /* Name of file containing the message */
   char from [30];       /* The sender of the message */
   char subject [36];    /* Can you guess what this is? */
   char date [16];       /* The time and date the message was despatched */
   INT_16 unused;        /* Not currently used. */
   UCHAR colour;         /* Display colour for this entry */
   char unused_2;        /* Not currently used */
   long mtime;           /* Binary version of message date for sorting */
   long fsize;           /* DOS size of this message (*The ^z separating
                            messages is not a part of this count) */
   } IMESSAGE;

As with the master file headers, Intel word order is used.

The "flags" field is a bitmap which contains information about the
message; the following bit values are possible:

#define FILE_MAILED            1   // IMESSAGE flag indicates mailed file
/* (*FILE_MAILED is set/cleared by "has local-style attachments" checkbox) */
#define UUENCODED              2   // "   "   "   "   "   "   uuencoding
#define ENCRYPTED              4   // "   "   "   "   "   "   encryption
#define DIGEST                 8   // (*) The message is a digest
#define EXPIRED             0x10   // The message is past its expiry date
#define FILE_ASCII          0x20   // (#) Attachment flag indicates ASCII file
#define HAS_BEEN_READ       0x80   // Indicates that an IMESSAGE has been read
#define HAS_ALT_VERS       0x100   // (*) Message has alternative versions
#define HAS_HTML           0x200   // (*) Message contains HTML data
#define CONFIRMATION      0x2000   // (#) Sender wants confirmation of reading
#define CONTENT_CTRL      0x4000   // (*) Message has undergone CC Processing
#define COPYSELF         0x20000L  // The message is a copy to self
#define DELETED          0x40000L  // (#) The message has been deleted.
#define MIME             0x80000L  // The message is a MIME transmission
#define REPLIED         0x100000L  // The message has been replied to.
#define FORWARDED       0x200000L  // The message has been forwarded.
#define URGENT          0x400000L  // The message is urgent/high priority.
#define BINHEX          0x800000L  // The message is a BinHex file
#define IS_MHS         0x1000000L  // The message originated from MHS
#define IS_SMTP        0x2000000L  // The message originated via SMTP
#define IS_ANNOTATED   0x4000000L  // The message has an annotation
#define ENCLOSURE      0x8000000L  // The message has enclosures
#define HIGHLIGHTED   0x10000000L  // The message has transient significance
#define MIME_MULTI    0x20000000L  // The message is in MIME Multipart format
#define TEXT_ENRICHED 0x40000000L  // The message is in "text/enriched" format
#define READ_ONLY     0x80000000L  // The message may not be deleted

All unused bit values are strictly reserved - you must not attempt to
set any other bit in the mask.

"fpos" should represent the offset of the first byte of the message in
the master file (Intel order). (*Relative to zero)

"msg_number" should be an index value into the index file at which this
IMESSAGE structure can be found. It is used as a quick way of calculating
where the IMESSAGE is in case it needs to be updated. (*Relative to zero)

"fname" is used in new mail to contain the filename of the message. For
mail in folders it can have any value so long as it is unique (usually it
is best to leave it as the name of the original new mail file).

"from", "subject" and "date" contain excerpted versions of these fields
from the message, and are used only for screen display in the browser.
You should NEVER attempt to generate replies based on these fields. In
the event that they contain high-bit characters, these fields will
contain them in the IBM-437 character set: Macintosh and Windows clients
are responsible for ensuring that characters are stored in this format as
required.

 The mtime posting in this thread is the latest  version.

 

---

Han van den Bogaerde - support@vandenbogaerde.net
Member of Pegasus Mail Support Group.
My own Pegasus Mail related web information:
http://www.vandenbogaerde.net/pegasusmail/

Ah, thank you (Dank U) Han.... exactly what I needed.

 

Could you clarify the annotation issue?  Should I move that to another thread? 

 

I would like to convert the old annotations to the new format so they can be seen in the message as before, and I also would like a way to find the annotation file given the message in the PMI/PMM file.  If you want I can email you directly if this is a secret for any reason. 

Robert_Lema:

 

Could you clarify the annotation issue?  Should I move that to another thread? I would like to convert the old annotations to the new format so they can be seen in the message as before, and I also would like a way to find the annotation file given the message in the PMI/PMM file.  If you want I can email you directly if this is a secret for any reason. 

I really have no idea how the relation between messages and annotations is kept? Sorry.

 

---

Han van den Bogaerde - support@vandenbogaerde.net
Member of Pegasus Mail Support Group.
My own Pegasus Mail related web information:
http://www.vandenbogaerde.net/pegasusmail/

I have opened a thread at http://community.pmail.com/forums/thread/6239.aspx in the Technical Support section as a bug report/request for information.

Hopefully David may be willing/able to shed some light on this?