Community Discussions and Support
Multiple copies of email from POP3 download

 

We've just started using Mecury to download from two POP 3 Domain mailboxes and distribute mail to local users. The clients use Pegasus Mail. However, with large emails we get multiple copies of the same email but of incrementally larger size. Eventually the whole message will come through but split up into many bits. We could increase the polling time but this would have to be set to the largest anticipated email. We used to use Pegasus to do the downloads and distribute the email with filtering and never had this happen. Mercury32 is installed on a NAS Windows Server 2003 R2 Appliance Edition.

 Any help would be greatly appreciated.

 Dale Trimble

<P mce_keep="true"> </P> <P>We've just started using Mecury to download from two POP 3 Domain mailboxes and distribute mail to local users. The clients use Pegasus Mail. However, with large emails we get multiple copies of the same email but of incrementally larger size. Eventually the whole message will come through but split up into many bits. We could increase the polling time but this would have to be set to the largest anticipated email. We used to use Pegasus to do the downloads and distribute the email with filtering and never had this happen. Mercury32 is installed on a NAS Windows Server 2003 R2 Appliance Edition.</P> <P> Any help would be greatly appreciated.</P> <P> Dale Trimble</P>

[quote user="dalet"]

 

We've just started using Mecury to download from two POP 3 Domain mailboxes and distribute mail to local users. The clients use Pegasus Mail. However, with large emails we get multiple copies of the same email but of incrementally larger size. Eventually the whole message will come through but split up into many bits. We could increase the polling time but this would have to be set to the largest anticipated email. We used to use Pegasus to do the downloads and distribute the email with filtering and never had this happen. Mercury32 is installed on a NAS Windows Server 2003 R2 Appliance Edition.

 Any help would be greatly appreciated.

 Dale Trimble

[/quote]

 

Pegasus Mail is checking the mail as it is being downloaded and sees this message in the inbox.  You can increase the new mail folder polling time for the new mail so this does not happen.  That said, this is exactly what Mercury/32 using MercuryD is designed to do and Mercury/32 has an option to not process the mail until it's the same size on two separate polls.  I recommend you use Mercury to perform this task.

 

[quote user="dalet"]<p mce_keep="true"> </p> <p>We've just started using Mecury to download from two POP 3 Domain mailboxes and distribute mail to local users. The clients use Pegasus Mail. However, with large emails we get multiple copies of the same email but of incrementally larger size. Eventually the whole message will come through but split up into many bits. We could increase the polling time but this would have to be set to the largest anticipated email. We used to use Pegasus to do the downloads and distribute the email with filtering and never had this happen. Mercury32 is installed on a NAS Windows Server 2003 R2 Appliance Edition.</p> <p> Any help would be greatly appreciated.</p> <p> Dale Trimble</p><p>[/quote]</p><p> </p><p>Pegasus Mail is checking the mail as it is being downloaded and sees this message in the inbox.  You can increase the new mail folder polling time for the new mail so this does not happen.  That said, this is exactly what Mercury/32 using MercuryD is designed to do and Mercury/32 has an option to not process the mail until it's the same size on two separate polls.  I recommend you use Mercury to perform this task. </p><p> </p>

Hi Thanks for replying

 Can you tell me where these polling controls are located in the configuration options in Mercury32. I've had a look but am unable to see them.

Cheers! 

Dale

 

<P>Hi Thanks for replying</P> <P> Can you tell me where these polling controls are located in the configuration options in Mercury32. I've had a look but am unable to see them.</P> <P>Cheers! </P> <P>Dale</P> <P mce_keep="true"> </P>

[quote user="dalet"]

Hi Thanks for replying

 Can you tell me where these polling controls are located in the configuration options in Mercury32. I've had a look but am unable to see them.

Cheers! 

Dale

 

[/quote]

 

Configuration | Mercury core |  Mail queue | Process the queue in two passes (file locking) 


 

[quote user="dalet"]<p>Hi Thanks for replying</p> <p> Can you tell me where these polling controls are located in the configuration options in Mercury32. I've had a look but am unable to see them.</p> <p>Cheers! </p> <p>Dale</p> <p mce_keep="true"> </p><p>[/quote]</p><p> </p><p>Configuration | Mercury core |  Mail queue | Process the queue in two passes (file locking)  </p><p> </p><p> </p>

 

 Thanks again

However, this option is already ticked. It's when Mercury is downloading from the remote POP3 box that the multiple file downloads occur. Is there a way of configuring MercuryD to do the same?

Many thanks

 

Dale

<P mce_keep="true"> </P> <P> Thanks again</P> <P>However, this option is already ticked. It's when Mercury is downloading from the remote POP3 box that the multiple file downloads occur. Is there a way of configuring MercuryD to do the same?</P> <P>Many thanks</P> <P mce_keep="true"> </P> <P>Dale</P>

[quote user="dalet"]

 

 Thanks again

However, this option is already ticked. It's when Mercury is downloading from the remote POP3 box that the multiple file downloads occur. Is there a way of configuring MercuryD to do the same?

Many thanks

 

Dale

[/quote]

 

This is really strange.  MercuryD should never be processing a file for delivery multiple times during download.  How often are you scanning the new mail folder in Pegasus Mail?  How log does it take to write a message?   It should only take seconds to write even a large file and if you are rescanning the new mail folder every 30 seconds or so you should never see a partially written *.CNM file.  How are you actually setup in MercuryD to accept the files?  If the username is blank and the default user is the user then the mail should be processed by core for delivery to the user.  You might need an alias to a local SMTP email address for this but delivery by core is a lot better than delivery by mercuryD.

 

 

[quote user="dalet"]<p mce_keep="true"> </p> <p> Thanks again</p> <p>However, this option is already ticked. It's when Mercury is downloading from the remote POP3 box that the multiple file downloads occur. Is there a way of configuring MercuryD to do the same?</p> <p>Many thanks</p> <p mce_keep="true"> </p> <p>Dale</p><p>[/quote]</p><p> </p><p>This is really strange.  MercuryD should never be processing a file for delivery multiple times during download.  How often are you scanning the new mail folder in Pegasus Mail?  How log does it take to write a message?   It should only take seconds to write even a large file and if you are rescanning the new mail folder every 30 seconds or so you should never see a partially written *.CNM file.  How are you actually setup in MercuryD to accept the files?  If the username is blank and the default user is the user then the mail should be processed by core for delivery to the user.  You might need an alias to a local SMTP email address for this but delivery by core is a lot better than delivery by mercuryD.</p><p> </p><p> </p>

 

 

Hello Again

The multiple file downloading occurs during the POP3 download and can be seen happening in the Mercury D window. Every time there is a poll a section of the email is downloaded but in ever increasing size. Presumably the core then passes the partially processed mail to the Pegasus mailboxes and the user sees all of the partial messages. We've increased the poll time to two minutes but this won't stop it from retrieving a partially processed message presumably. I realise that we could make the interval much longer but this doesn't seem to be a solution.  Otherwise the integrated Mercury/Pegasus setup works fine and seems straightforward. When you ask how we are setup in MercuryD to accept files I'm a bit unclear what you mean. Basically Mercury collects the mail from a Domain Box POP3 mailbox and routes the mail to the mailboxes. I could supply settings details if that might help.

Dale 

<P mce_keep="true"> </P> <P mce_keep="true"> </P> <P>Hello Again</P> <P>The multiple file downloading occurs during the POP3 download and can be seen happening in the Mercury D window. Every time there is a poll a section of the email is downloaded but in ever increasing size. Presumably the core then passes the partially processed mail to the Pegasus mailboxes and the user sees all of the partial messages. We've increased the poll time to two minutes but this won't stop it from retrieving a partially processed message presumably. I realise that we could make the interval much longer but this doesn't seem to be a solution.  Otherwise the integrated Mercury/Pegasus setup works fine and seems straightforward. When you ask how we are setup in MercuryD to accept files I'm a bit unclear what you mean. Basically Mercury collects the mail from a Domain Box POP3 mailbox and routes the mail to the mailboxes. I could supply settings details if that might help.</P> <P>Dale </P>

 

I've just had another go at this and I think it is probably solved. By increasing the scanning interval of the new mail folder by Pegasus, larger messages are now coming in as a single message. I can't really grasp why this should  happen in the first place but it seems to work and the increase in scanning time is no sacrifice. Why are the new mail files rewritten?

Anyway, thanks for the solution and the help.

Dale

<P mce_keep="true"> </P> <P>I've just had another go at this and I think it is probably solved. By increasing the scanning interval of the new mail folder by Pegasus, larger messages are now coming in as a single message. I can't really grasp why this should  happen in the first place but it seems to work and the increase in scanning time is no sacrifice. Why are the new mail files rewritten? </P> <P>Anyway, thanks for the solution and the help.</P> <P>Dale</P>

[quote user="dalet"]

 

The file gets 'dissected' during the POP3 download by Mercury. Viewing the MercuryD window the message can be seen being 'retrieved'  but in ever larger chunks. This also apparent in the log file where the identical message is downloaded multiple times but then processed separately. This seems to happen well before the core process passed the messages to the Pegasus mailboxes. I'm not sure what you mean about our setup on Mercury and the usernames but will supply any info that will help.

 

Thanks

Dale

[/quote]

 

Very strange.  There should be no reason at all that the message is being downloaded like that unless there is some sort of failure during the download.  Even then the message should be discarded rather then saved to the users new mail directory.  Have you run a session log on MercuryD to see exactly what's happening with the downloads?

 

[quote user="dalet"]<p mce_keep="true"> </p> <p>The file gets 'dissected' during the POP3 download by Mercury. Viewing the MercuryD window the message can be seen being 'retrieved'  but in ever larger chunks. This also apparent in the log file where the identical message is downloaded multiple times but then processed separately. This seems to happen well before the core process passed the messages to the Pegasus mailboxes. I'm not sure what you mean about our setup on Mercury and the usernames but will supply any info that will help.</p> <p mce_keep="true"> </p> <p>Thanks </p> <p>Dale</p><p>[/quote]</p><p> </p><p>Very strange.  There should be no reason at all that the message is being downloaded like that unless there is some sort of failure during the download.  Even then the message should be discarded rather then saved to the users new mail directory.  Have you run a session log on MercuryD to see exactly what's happening with the downloads?</p><p> </p>

 

Thomas

 

Seems like I managed to mix up my replies somehow. The problem is definitely related to the scan time of the new mail folder by Pegasus. Increase the scanning time and the message splits are fewer and the messages larger. Close Pegasus down and the message comes in whole. I can't fathom this. Is it possible that this is related to read\write access to the folder by Mecury? I've checked the account from which Mercury is logged in but full control is specified. Any ideas?

 Many thanks

 

Dale

<P mce_keep="true"> </P> <P>Thomas</P> <P mce_keep="true"> </P> <P>Seems like I managed to mix up my replies somehow. The problem is definitely related to the scan time of the new mail folder by Pegasus. Increase the scanning time and the message splits are fewer and the messages larger. Close Pegasus down and the message comes in whole. I can't fathom this. Is it possible that this is related to read\write access to the folder by Mecury? I've checked the account from which Mercury is logged in but full control is specified. Any ideas?</P> <P> Many thanks</P> <P mce_keep="true"> </P> <P>Dale</P>

What new mail scan time are you using?  What size messages are these that it takes multiple scan times of the new mail folder before they are downloaded?

 

<p>What new mail scan time are you using?  What size messages are these that it takes multiple scan times of the new mail folder before they are downloaded?</p><p> </p>

 

 

The original scan time was just ten seconds but this has never mattered before (pre Mercury). This meant that with a 1 to 1.5mb message you got three emails, two small ones of 0.4m - 0.7mb and the final one of the correct size. Increase this to 30 seconds and the mail comes in whole. I understand that I can increase this to much longer but what happens if Pegasus scans mid way through a large message still writing to disk? 

 Thanks

 Dale 

<P mce_keep="true"> </P> <P mce_keep="true"> </P> <P>The original scan time was just ten seconds but this has never mattered before (pre Mercury). This meant that with a 1 to 1.5mb message you got three emails, two small ones of 0.4m - 0.7mb and the final one of the correct size. Increase this to 30 seconds and the mail comes in whole. I understand that I can increase this to much longer but what happens if Pegasus scans mid way through a large message still writing to disk? </P> <P> Thanks</P> <P> Dale </P>

[quote user="dalet"]

 

 

The original scan time was just ten seconds but this has never mattered before (pre Mercury). This meant that with a 1 to 1.5mb message you got three emails, two small ones of 0.4m - 0.7mb and the final one of the correct size. Increase this to 30 seconds and the mail comes in whole. I understand that I can increase this to much longer but what happens if Pegasus scans mid way through a large message still writing to disk? 

 Thanks

 Dale 

[/quote]

 

It should have since the download via POP3 should take ma lot longer than mercury core writing a cnm file to the disk.  Again, how long is it taking to write these files to the new mail directory?  A one MByte message getting you three separate messages in the new mail folder  means that it's taking over 30 seconds to write a 1 MByte file.  I guess I really do not understand what you are doing with your setup.  I've set my re-scan of the new mail folder to 3 seconds and never see this problem.

 

 

[quote user="dalet"]<p mce_keep="true"> </p> <p mce_keep="true"> </p> <p>The original scan time was just ten seconds but this has never mattered before (pre Mercury). This meant that with a 1 to 1.5mb message you got three emails, two small ones of 0.4m - 0.7mb and the final one of the correct size. Increase this to 30 seconds and the mail comes in whole. I understand that I can increase this to much longer but what happens if Pegasus scans mid way through a large message still writing to disk? </p> <p> Thanks</p> <p> Dale </p><p>[/quote]</p><p> </p><p>It should have since the download via POP3 should take ma lot longer than mercury core writing a cnm file to the disk.  Again, how long is it taking to write these files to the new mail directory?  A one MByte message getting you three separate messages in the new mail folder  means that it's taking over 30 seconds to write a 1 MByte file.  I guess I really do not understand what you are doing with your setup.  I've set my re-scan of the new mail folder to 3 seconds and never see this problem. </p><p> </p><p> </p>

Hi

 Apologies for the late reply. My colleague and I have been busy with this and other matters.

 The downloaded data is being written to a RAID5 array comprising 4 fast SATA disks in a NAS device. The same array is used to store our working data and we have not experienced any delays reading/writing/copying to/from the array during normal usuage. Our connection is a 0.5mbit broadband connection, so large mails will take a short while to download.

 Mercury is configured as follows:

Core Module Poll Time: 10 seconds

MercuryD POP3 Module Poll Time: 180 seconds

Mercury is running under the local Administrator account

I recently sent a 7MB attachment from a web account to my address, and after delivery the mail message was fragmented, with each fragment larger then the one before until the last message contains the complete file. So, the result is that I received several emails, of progressively larger sizes, and only the last message contained the complete attachment.

 We cannot understand what is happening here. It looks as though the Core Module is forwarding the mail message before all the file has been downloaded. Presumably this cannot happen as the file would be flagged as incomplete and not ready for forwarding.

 If you or anyone else has any thoughts on this we will be very grateful to hear them.

Thanks

<P>Hi</P> <P> Apologies for the late reply. My colleague and I have been busy with this and other matters.</P> <P> The downloaded data is being written to a RAID5 array comprising 4 fast SATA disks in a NAS device. The same array is used to store our working data and we have not experienced any delays reading/writing/copying to/from the array during normal usuage. Our connection is a 0.5mbit broadband connection, so large mails will take a short while to download.</P> <P> Mercury is configured as follows:</P> <P>Core Module Poll Time: 10 seconds</P> <P>MercuryD POP3 Module Poll Time: 180 seconds</P> <P>Mercury is running under the local Administrator account</P> <P>I recently sent a 7MB attachment from a web account to my address, and after delivery the mail message was fragmented, with each fragment larger then the one before until the last message contains the complete file. So, the result is that I received several emails, of progressively larger sizes, and only the last message contained the complete attachment.</P> <P> We cannot understand what is happening here. It looks as though the Core Module is forwarding the mail message before all the file has been downloaded. Presumably this cannot happen as the file would be flagged as incomplete and not ready for forwarding.</P> <P> If you or anyone else has any thoughts on this we will be very grateful to hear them.</P> <P>Thanks</P>

I have just sent the same attachment which has been delivered directly to my Pegasus mailbox by the SMTP module (not via POP3) and the file arrived in one piece.

Is there a problem with the POP3 module/Mercury Core Module? This seems very strange.

<P>I have just sent the same attachment which has been delivered directly to my Pegasus mailbox by the SMTP module (not via POP3) and the file arrived in one piece.</P> <P>Is there a problem with the POP3 module/Mercury Core Module? This seems very strange.</P>

[quote user="Greenman"]

I have just sent the same attachment which has been delivered directly to my Pegasus mailbox by the SMTP module (not via POP3) and the file arrived in one piece.

Is there a problem with the POP3 module/Mercury Core Module? This seems very strange.

[/quote]

 

No it is not delivered by the SMTP module, it is placed in the queue as a MG*.QCF/QDF file pair and it's delivered by core.  In same way, if you leave the local user blank in MercuryD and  and use an alias to a local domain for this user then it's going to be delivered by core as well.

[quote user="Greenman"]<p>I have just sent the same attachment which has been delivered directly to my Pegasus mailbox by the SMTP module (not via POP3) and the file arrived in one piece.</p> <p>Is there a problem with the POP3 module/Mercury Core Module? This seems very strange.</p><p>[/quote]</p><p> </p><p>No it is not delivered by the SMTP module, it is placed in the queue as a MG*.QCF/QDF file pair and it's delivered by core.  In same way, if you leave the local user blank in MercuryD and  and use an alias to a local domain for this user then it's going to be delivered by core as well. </p>

Hi Thomas, thanks again for taking the time to reply.

What I don't understand is when mail to our organisation was being delivered via POP3, messages with attachments were delivered to user's inboxes in fragments. However, when the same messages are delivered via SMTP the user receives a single message.

The way Mercury32 delivers these messages to the user's inbox does, in our experience, vary depending on whether the POP3 module or the SMTP (both the normal server and the end-to-end server) is used.

If you want any more information about how our installation of Mercury32 is configured, please ask.

 

<P>Hi Thomas, thanks again for taking the time to reply.</P> <P>What I don't understand is when mail to our organisation was being delivered via POP3, messages with attachments were delivered to user's inboxes in fragments. However, when the same messages are delivered via SMTP the user receives a single message.</P> <P>The way Mercury32 delivers these messages to the user's inbox does, in our experience, vary depending on whether the POP3 module or the SMTP (both the normal server and the end-to-end server) is used.</P> <P>If you want any more information about how our installation of Mercury32 is configured, please ask.</P> <P mce_keep="true"> </P>

[quote user="Greenman"]

Hi Thomas, thanks again for taking the time to reply.

What I don't understand is when mail to our organisation was being delivered via POP3, messages with attachments were delivered to user's inboxes in fragments. However, when the same messages are delivered via SMTP the user receives a single message.

The way Mercury32 delivers these messages to the user's inbox does, in our experience, vary depending on whether the POP3 module or the SMTP (both the normal server and the end-to-end server) is used.

If you want any more information about how our installation of Mercury32 is configured, please ask.

 

[/quote]

 

Have you entered a username for the POP3 download?  If so leave it blank and use either the header in the message or an alias to deliever the mail.  In this way it's going to be passed to mercury core for delivery after it has been completely downloaded from the mail server.

 

[quote user="Greenman"]<p>Hi Thomas, thanks again for taking the time to reply.</p> <p>What I don't understand is when mail to our organisation was being delivered via POP3, messages with attachments were delivered to user's inboxes in fragments. However, when the same messages are delivered via SMTP the user receives a single message.</p> <p>The way Mercury32 delivers these messages to the user's inbox does, in our experience, vary depending on whether the POP3 module or the SMTP (both the normal server and the end-to-end server) is used.</p> <p>If you want any more information about how our installation of Mercury32 is configured, please ask.</p> <p mce_keep="true"> </p><p>[/quote]</p><p> </p><p>Have you entered a username for the POP3 download?  If so leave it blank and use either the header in the message or an alias to deliever the mail.  In this way it's going to be passed to mercury core for delivery after it has been completely downloaded from the mail server.</p><p> </p>

hello everybody

this is my first reply ever and i joined the forum only to say that i had the same problem and i fixed it by creating a temp folder where incoming messages are put when delivered, (and once complete then moved to the users mail folder) put the user default name to be the acct that mail is delivered and... voila.. no more multiple copies. 

<p>hello everybody</p><p>this is my first reply ever and i joined the forum only to say that i had the same problem and i fixed it by creating a temp folder where incoming messages are put when delivered, (and once complete then moved to the users mail folder) put the user default name to be the acct that mail is delivered and... voila.. no more multiple copies. </p>
live preview
enter atleast 10 characters
WARNING: You mentioned %MENTIONS%, but they cannot see this message and will not be notified
Saving...
Saved
With selected deselect posts show selected posts
All posts under this topic will be deleted ?
Cancel
Delete
Pending draft ... Click to resume editing
Discard draft