Yes for Barius .  And I get errors I don't know how to correct.  space between pattern & following words; missing operators, barewood where operator expected, etc. 

The C++ program contains no EXE  .  Don't know what to do with it.

Background:  User running Pegasus v4.41 with a >8gb mailbox. 

WinNT5.1 SP2 ("XP SP2") on FAT32.  Multiple mail folders store

legally-important information back to 1997(?).  No individual mail

folder should be exceeding 1.5gb at this time.

Our only Pegasus user (in another state, sadly) is experiencing index corruption in her mailbox.  She reports a variety of symptoms:

1. Opening a message from the in-folder list ("Foo" from "Bar", body "abcd") returns the wrong message ("Apples" from "Fred", body "1234").
2. Opening a message returns nothing (i.e. blank e-mail)
3. Opening a message returns hex garbage (i.e. middle of an attachment in another e-mail)
4. Pegasus complains that the indexes may be corrupted
   

She has re-indexed but has not confirmed if it helped at all.  The problems are continuing.

What is causing this?  Can it be corrected?  Is there a way to tell if the raw folder file is still OK?  Have we pushed Pegasus too far?  What other information would be helpful in diagnosing or correcting the problem?  Am I asking too many questions?

This user, above all others in this company, needs her e-mail to be rock-solid.  Are there any recommendations from the community for improving the arrangement?  I'm giving serious thought to moving her data to an IMAP server and letting Pegasus cache it.

 
Daniel Johnson
Computer Resources
C. H. Guernsey & Company

 

[quote user="chguernsey"]

Background:  User running Pegasus v4.41 with a >8gb mailbox. 

WinNT5.1 SP2 ("XP SP2") on FAT32.  Multiple mail folders store

legally-important information back to 1997(?).  No individual mail

folder should be exceeding 1.5gb at this time.

Should not be any particular problem with the size of the folders or total size of the mailbox. Mine's a lot larger, many others  are as well. 
 

Our only Pegasus user (in another state, sadly) is experiencing index corruption in her mailbox.  She reports a variety of symptoms:

1. Opening a message from the in-folder list ("Foo" from "Bar", body "abcd") returns the wrong message ("Apples" from "Fred", body "1234").
2. Opening a message returns nothing (i.e. blank e-mail)
3. Opening a message returns hex garbage (i.e. middle of an attachment in another e-mail)
4. Pegasus complains that the indexes may be corrupted

If Pegasus Mail complains about the indexes they are probably corrupted.  There is fact may be something wrong with a particular folder as well.  Here's a way to fix a folder when the re-indexing does not solve the problem.

1.  Move all the messages you want to save to a temp folder and
    delete all the messages in the original folder.

2.  Use the right click "recover deleted space" to remove all
    messages form the folder.

3.  If step 2 works then simply move all of the messages from the
    temp folder back to the original folder and delete the temp
    folder.

4.  If step 2 fails then delete the original folder and rename the
    temp folder to match the original name.  Reset the filter
    pointing to this folder.

She has re-indexed but has not confirmed if it helped at all.  The problems are continuing.

What is causing this?  Can it be corrected?  Is there a way to tell if the raw folder file is still OK?  Have we pushed Pegasus too far?  What other information would be helpful in diagnosing or correcting the problem?  Am I asking too many questions?

 Personally I alway suspect that something is fighting WinPMail for the folder access.  The first thing that come to mind and one that can always cause this sort of problem is an anti-virus program with auto protect turned on grabbing the files when WinPMail accesses them.  The a-v program should not be working the WinPMail directories; the viruses should either be caught at the mail server or they should be using the extension VirScan to check the mail when read.  This extension will use most any a-v software.

This user, above all others in this company, needs her e-mail to be

rock-solid.  Are there any recommendations from the community for

improving the arrangement?  I'm giving serious thought to moving her

data to an IMAP server and letting Pegasus cache it.

If these data are so critical to the company then moving it to IMAP4 might be a good idea anyway.  This way you'll have all the data on your server where it will be protected and backed up on a regular basis.   If anything goes wrong you'll always be able to restore it from your daily backup.

 
Daniel Johnson
Computer Resources
C. H. Guernsey & Company

Problem #1 and #3 are likely the result of that particular email

'folder' (actually the underlying .pmm file) growing beyond 2Gb in

size.  Pegasus doesn't seem to be able to cope with files larger than

2Gb on 32bit Windows.  Tell your employee/client/whatever that she needs to break up her email into more folders so they don't grow too big.

Problem #1 and  #2 are sometimes caused by having more than 65k emails in a single folder.  See above.

Problem

#4 can be fixed by selecting and moving/copying all emails from the

corrupted folder to a new folder.  Then delete the original folder and

rename the new one.  NOTE BENE: if the .pmm file for this folder is

over the 2Gb limit, then this *will* result in loss of data.  The only

way I have found to recover such files is to use a perl script that

splits the file into separate emails, then move those emails back into the

Pegasus newmail folder (in Windows Explorer) and then use Pegasus to move them into multiple mail folders.  The perl script I wrote is below, just copy it into a text file (e.g. pmmsplit.pl) and call it as so: pmmsplit.pl <input file path.pmm> <output filepath/prefix>

Note that you will need to install Perl if you are running Windows.  If you happen to have a Linux or Mac machine around, Perl should be installed there by default. 

------------------------- 

#!/usr/bin/perl

$thefile = $ARGV[0];
$outfile = $ARGV[1];

if(!$thefile || !$outfile){
        die('Usage: pmmsplit.pl < input file path/file.pmm > < output directory path/prefix >');
}

open (READ,"<$thefile") || die('The input file cannot be opened');
$count = 0;
while(<READ>){ //Loop through each line of the file
        if(/^Z/){
                //if this line contains '^Z' then it is a boundary between emails.
                //First, split the line using ^Z to get any data that might be on the same line
                @linesplit = split(/^Z/,$_);
                if($#linesplit > 1){
                         //there was some data before the ^Z, print it out to file.
                        print OUT @linesplit[0];
                }
                close(OUT);//Close the current output file, it now contains a complete MIME message (assuming a ^Z delimiter)

                $count=$count+1;//increment counter
                open(OUT, ">$outfile".$count.".cnm");//Open a new output file
                print OUT @linesplit[1];//Print any data that came after ^Z
        }
        else{
                print OUT $_;//This line does not contain the delimiter, just print it directly to the current output file.
        }
}

close(READ);
------------------------------

Thank you!  I now have a copy of her mail directory (FedEx and external HDs are nice).  There is one 2.5gb mail folder, the others are less than 2gb.  I've copied them to the mail server and am trying to split them out into Maildir-format.  I'm using the following script to call your Perl script.

#!/bin/bash
# "dosplit"
# Uses "dosplit.names", created by editing the output of
#      for MF in *.pmm; do echo "$MF `head -c 50 $MF`" >> dosplit.names; done
# to remove spaces from mail folder names.
MAILBASEPATH=~/Maildir/.zzDJTemp
cat dosplit.names | while read MFLINE; do
  MF=`echo $MFLINE | cut -f 1 -d " "`
  if [ ! -f $MF.split ]; then
    MFB=`basename $MF .pmm`
    MFNAME=`echo $MFLINE | cut -f 2 -d " "`
    mkdir -p $MAILBASEPATH.$MFNAME/cur
    mkdir -p $MAILBASEPATH.$MFNAME/new
    mkdir -p $MAILBASEPATH.$MFNAME/tmp
    DEST=$MAILBASEPATH.$MFNAME/tmp/$MFB-$$-
    date > $MF.split
    echo "Splitting $MF ($MFNAME) into $DEST"
    ./pmmsplit.pl $MF $DEST
    date >> $MF.split
  fi
done
#######

It creates tons of files, but many of them are just base64 chunks. At least one of the CNMs I opened has a base64 attachment chunk preceding the message, and the message's own attachment chunk is cut off.  Your code looks OK to me, but I'm not a Perl-guru.  Do you (or anyone else) have any ideas on how to fix this?  I'd much rather make this work than try to make my copy of Pegasus properly read her files.

As far as i can tell, all your script is supposed to do is split the separate emails into a folder structure of <mail folder name>/<cur | new | tmp>.  It appears to drop all of the emails into 'tmp', probably so that the email client can process them into 'cur'.  All that my script does is cut the .pmm file on every occurance of ^Z (ctrl-Z) which is used by Pegasus to separate the MIME messages.  I'm not sure if Pegasus can use another delimeter or not, if so then you'd have to open the .pmm file (on Linux try 'tail -n #### file.pmm' to read the last #### lines out) and find out what delimeter is being used.  Also note, I've never used my script on Windows, only Linux.  It is entirely possible that the same 2Gb limit that affects Pegasus on Windows will also affect my script...

I'm not sure why you want the emails in maildir format, are you planning on using some particular client that expects this?  If you need to move to another client I suggest Thunderbird for ease of use.  Moving to Tbird is fairly straight-forward and involves using Pegasus to create Unix Mail Format folders instead of the usual Pmail v2.x format folders.  The Unix Mail Format folders in Pegasus are directly transferable to Tbird.  Note, however, that Tbird can also run into the same 2Gb problem, as can Outlook (which will then unkindly force you to 'archive' old emails to clear up space).  If your client is used to Pegasus, I strongly suggest staying with it.  You might also file a bug-report on this 2Gb problem.  I filed one sometime back and was told it was a Windows problem, though I think it would be responsible to perform a platform based size-limit test within Pegasus to prevent overfilling a folder, and maybe if you complain too then someone will listen.

Edit: I added comments to my previous code so you might understand it better.
 

I recall splitting in a similar manner (using VB6) for a mass-printing application a few years ago.  I'm checking a hex-dump of one of the smaller PMMs to see if I can find anything interesting.  I'm doing all of this on the Linux server that will end up hosting the mail, the files haven't touched a Windows computer or Windows file system since they left her laptop.

The Courier IMAP server expects it in this case.  I plan to store all of her mail on the server, with the well-split Pegasus archives on her local PC for all but the current year.  Maildir allows me to bypass filesize limits and open/view/print/access the mail with any IMAP client.

She is used to Pegasus, and I will keep her on it if I can.  I've never been a fan of lookOut, though there is pressure in the company to "standardize her".

Aye, I may mention this. It's not the first time that we've had 2gb trouble with her, just the first time that it wasn't easily fixed.

Thank you for the code edits, it is clearer to me now. I'll post as soon as I find anything.

Got it at last! I wrote a quick C++ program to split the PMM file, and it worked.  I still don't see why the Perl program didn't.  :-/   Source code available, e-mail me at djohnson@chguernsey.com if you want it.

I apologize for the delay, several people have asked for this and I've just now gotten around to it.

http://thor.chguernsey.com/temp/split-sig.zip    12kb

MD5sum: 3f82ab2945b6eaec3211f5d74d9296c2 *split-sig.zip
The ZIP includes my PGP signature.  I'll keep the file available at that location until at least 2009-01-01.

The program was written in C++ and compiled on 32-bit Linux.  It should work as-is on Windows but I don't guarantee it.  I left some extra files in the archive that demonstrate how I actually used it, I hope it's helpful.

Daniel Johnson
Progman2000@usa.net
PGP key 0x163E81EF
djohnson@chguernsey.com

I liked Barius's approach to splitting up folders into messages, but found his Perl script to be overly complicated and extremely buggy. I've rewritten it as shown below. I tested it on a folder with 445 messages, and it took ~9 seconds to process.

#!/usr/bin/perl -w
use strict;
# pmmsplit.pl
#
# Grant Root <groot dayton-phoenix="" com=""></groot>
# 
# This is a rewritten version of a script by "Barius" at
# http:#community.pmail.com/forums/thread/4125.aspx
#
# This script will split a Pegasus Mail folder file into individual message files
# suitable for moving into the mailbox directory of a target user, where they will
# show up as messages in the New Mail folder.
#
# A prime use for this utility is when a folder has exceeded 2GB in size, and can no
# longer be properly accessed or reindexed through the Pegasus Mail interface.
my $infile = $ARGV[0];
my $prefix = $ARGV[1];
if(!$infile || !$prefix){
    die('Usage: pmmsplit.pl <input file="" path="" file.pmm=""><output directory="" path="" prefix="">');
</output>
}
open (IN,"<$infile") || die('The input file cannot be opened.');
my $header;
if (read (IN,$header,128) != 128){  # Read past the folder header.
    die('File does not appear to be a Pegasus Mail folder.');
}
my $count = 1;
open(OUT, ">$prefix".$count.".cnm");  # Open a new output file.
while (<in>){  #Loop through each line of the file
</in>
    if (/^\x{1A}/){  # If this line starts with a ^Z then it is a boundary between emails.
        close(OUT);  # Close the current output file, it now contains a complete message.
        if (length($_) > 1) {
            $count++;
            open(OUT, ">$prefix".$count.".cnm");  # Open a new output file.
            print OUT substr($_, 1);  # Print any data that came after ^Z.
        }        
    }
    else{
        print OUT $_;  # This line does not contain the delimiter, just print it.
    }
}
close(IN);

I have need of something to rescue my wife's Pegasus Mainn.pmm folder which has exceeded 2 GB after someone sent her a bunch of big pictures.
The effect is that mail messages with cewrtain subjects do not reflect that subject at all.  All messages are wrong.

Checking produces 'Seek Error in Master File'

So I tried moving mail out & the Pegasus program  mbxmain_ui.exe functions.  Did not correct problem.

Found Barius' & Grant Root's scripts & decided to try them after installing Active State's Perl on my Windows 7 Pro PC.

Using Grant Root's script, now get errors in script  'Use of uninitialized value $_ in pattern match (m//) at script.pl line 35' &
'Use of uninitialized value $_  at script.pl line 44'

I'm not a pearl at 'Perl' !  

Any suggestions how to fix script and/or correct problem ?

PS - If Pegasus only supports 32 bit presently & Win 7 is running 64 bit; don't we have a a fundamental problem
until Pegasus is converted to 64 bit addressing ?!?  I have been using Pegasus since early 90's & would really like to continue - safely !

Have you tried Barius' perl script in this thread, the program that Daniel linked to, or the 'Notsplit' program in this site's Downloads / Tools and Utilities section?