Cyrillic SPAM control

GordonM

posted Aug 17 '09 at 11:11 pm

Well, "triggered by a regular expression" in filtering seems to do the job. The only issue remaining seems to be to minimize false positives, e.g. if someone sends me an email discussing SPAM and includes the characters charset="koir-8". I think that this can largely be eliminated by knowing where these characters can appear on a line (there would be an unfortunate case if body text in a wanted e-mail included charset etc at the beginning of a line). There seem to be several variations, e.g. charset on a line with Content-Type, charset on a separate line preceded by a tab or spaces, and the charset name surrounded by double-quotes or not.

Gordon

Well, "triggered by a regular expression" in filtering seems to do the job.&nbsp; The only issue remaining seems to be to minimize false positives, e.g. if someone sends me an email discussing SPAM and includes the characters charset="koir-8".&nbsp; I think that this can largely be eliminated by knowing where these characters can appear on a line (there would be an unfortunate case if body text in a wanted e-mail included charset etc at the beginning of a line).&nbsp; There seem to be several variations, e.g. charset on a line with Content-Type, charset on a separate line preceded by a tab or spaces, and the charset name surrounded by double-quotes or not. Gordon &nbsp;

GordonM

posted Aug 10 '09 at 8:53 pm

As with many other people, I have noticed a large increase in SPAM containing cyrillic characters. I am wondering whether Content Control can handle this. I don't know how to write cyrillic characters from the keyboard (though maybe there are Alt key strokes to do this), however, I can paste cyrillic into the content control .dat file.

For example I could use a rule like:

if BODY MATCHES *и* weight 51

As "и" is a common Russian character, almost any cyrillic e-mail will contain it. Can I expect Content Control to work properly using this technique? My main concern is whether Content Control might interpret cyrillic characters as something from an English character-set, causing most of my vaild e-mails to be deleted. As I don't receive any valid cyrillic e-mails, I am happy that they are all deleted

Thank you

Gordon

P.S. Nothing to do with my question, but when I used the "и" character in the above text, word-wrap seemed to stop working in this post!

As with many other people, I have noticed a large increase in SPAM containing cyrillic characters.&nbsp; I am wondering whether Content Control can handle this.&nbsp; I don't know how to write cyrillic characters from the keyboard (though maybe there are Alt key strokes to do this), however, I can paste cyrillic into the content control .dat file. For example I could use a rule like:<PRE wrap="">if BODY MATCHES *и* weight 51</PRE> As "и" is a common Russian character, almost any cyrillic e-mail will contain it.&nbsp; Can I expect Content Control to work properly using this technique?&nbsp; My main concern is whether Content Control might interpret cyrillic characters as something from an English character-set, causing most of my vaild e-mails to be deleted.&nbsp; As I don't receive any valid cyrillic e-mails, I am happy that they are all deleted<PRE wrap="">Thank you</PRE><PRE wrap="">Gordon</PRE><PRE wrap="">P.S. Nothing to do with my question, but when I used the "и" character in the above text, word-wrap seemed to stop working in this post!</PRE>

Thomas R. Stephenson

posted Aug 10 '09 at 9:01 pm

You can trigger on the MIME type in the message headers to cause all messages using a Cyrillic MIME type to be tagged as spam.

You can trigger on the MIME type in the message headers to cause all messages using a Cyrillic MIME type to be tagged as spam.&nbsp;

GordonM

posted Aug 10 '09 at 10:25 pm

Thank you, Thomas. I presume that this will be

Content-Type: text/plain; charset="koi8-r"

Gordon

Thank you, Thomas.&nbsp; I presume that this will be <BLOCKQUOTE><PRE id=line1>Content-Type: text/plain; charset="koi8-r"</PRE></BLOCKQUOTE><PRE>Gordon</PRE>

tBB

posted Aug 11 '09 at 9:48 am

Checking the Content-Type header alone doesn't help much. The following CC rules hit on Japanese, Russian and Chinese encodings:

# Content-Type

IF HEADER "Content-Type" MATCHES "/b*charset*iso-2022-jp*" WEIGHT 101
IF HEADER "Content-Type" MATCHES "/b*charset*koi8-*" WEIGHT 101
IF HEADER "Content-Type" MATCHES "/b*windows-1251*" WEIGHT 101
IF HEADER "Content-Type" MATCHES "/b*charset*Big5*" WEIGHT 101
IF HEADER "Content-Type" MATCHES "/b*charset*GB2312*" WEIGHT 101
IF HEADER "Content-Type" MATCHES "/b*charset*GB18030*" WEIGHT 101

# Subject

IF HEADER "Subject" MATCHES "*iso-2022-jp*" WEIGHT 101
IF HEADER "Subject" MATCHES "*koi8-*" WEIGHT 101
IF HEADER "Subject" MATCHES "*windows-1251*" WEIGHT 101
IF HEADER "Subject" MATCHES "*Big5*" WEIGHT 101
IF HEADER "Subject" MATCHES "*GB2312*" WEIGHT 101
IF HEADER "Subject" MATCHES "*GB18030*" WEIGHT 101

# Sender
IF SENDER CONTAINS "iso-2022-jp" WEIGHT 101
IF SENDER CONTAINS "koi8-" WEIGHT 101
IF SENDER CONTAINS "windows-1251" WEIGHT 101
IF SENDER CONTAINS "Big5" WEIGHT 101
IF SENDER CONTAINS "GB2312" WEIGHT 101
IF SENDER CONTAINS "GB18030" WEIGHT 101

Best regards,

Nico

Checking the Content-Type header alone doesn't help much. The following CC rules hit on Japanese, Russian and Chinese encodings: # Content-Type IF HEADER "Content-Type" MATCHES "/b*charset*iso-2022-jp*" WEIGHT 101 IF HEADER "Content-Type" MATCHES "/b*charset*koi8-*" WEIGHT 101 IF HEADER "Content-Type" MATCHES "/b*windows-1251*" WEIGHT 101 IF HEADER "Content-Type" MATCHES "/b*charset*Big5*" WEIGHT 101 IF HEADER "Content-Type" MATCHES "/b*charset*GB2312*" WEIGHT 101 IF HEADER "Content-Type" MATCHES "/b*charset*GB18030*" WEIGHT 101 # Subject IF HEADER "Subject" MATCHES "*iso-2022-jp*" WEIGHT 101 IF HEADER "Subject" MATCHES "*koi8-*" WEIGHT 101 IF HEADER "Subject" MATCHES "*windows-1251*" WEIGHT 101 IF HEADER "Subject" MATCHES "*Big5*" WEIGHT 101 IF HEADER "Subject" MATCHES "*GB2312*" WEIGHT 101 IF HEADER "Subject" MATCHES "*GB18030*" WEIGHT 101 # Sender IF SENDER CONTAINS "iso-2022-jp" WEIGHT 101 IF SENDER CONTAINS "koi8-" WEIGHT 101 IF SENDER CONTAINS "windows-1251" WEIGHT 101 IF SENDER CONTAINS "Big5" WEIGHT 101 IF SENDER CONTAINS "GB2312" WEIGHT 101 IF SENDER CONTAINS "GB18030" WEIGHT 101 Best regards, Nico

GordonM

posted Aug 12 '09 at 10:24 pm

Nico - Thank you for the examples. It's mostly Russian that I am bothered with at the moment, but I'll add in all the rules that you have provided.

Thank you

Gordon

Nico - Thank you for the examples.&nbsp; It's mostly Russian that I am bothered with at the moment, but I'll add in all&nbsp;the rules that you have provided. Thank you Gordon

GordonM

posted Aug 13 '09 at 2:32 pm

Nico - Adding the rulles that you suggested seems to make a significant difference. However, there are still some "Content-Type" headers not being detected. These are of the form:

Content-Type: text/plain;
	charset="koi8-r"

The charset is not on the same line as the "Content-Type" header ... there is a newline character plus a tab between them. Have you seen this as well?

In examing the Spam which is still getting through, most of it seems to have this characteristic.

There are also cases where there is more than one charset type header, the first is often Western European, but then there is another which is Eastern European.

If this could be detected, it might be a usefuil filtering approach.

I also notice that there is a difference in one of the header lines:

IF HEADER "Content-Type" MATCHES "/b*charset*koi8-*" WEIGHT 101
IF HEADER "Content-Type" MATCHES "/b*windows-1251*" WEIGHT 101
i.e. no charset for the windows-1251 line.

I must admit that some of the content control rules confuse me. I am not sure how useful the /b is and, maybe without it, the headers with the newline and tab in them would be caught.

Gordon

Nico - Adding the rulles that you suggested seems to make a significant difference.&nbsp; However, there&nbsp;are still some "Content-Type" headers not being detected.&nbsp; These are of the form: <pre id="line1">Content-Type: text/plain; charset="koi8-r"</pre> The charset is not on the same line as the "Content-Type" header ... there is a newline character plus a tab between them. Have you seen this as well? In examing the Spam which is still getting through, most of it seems to have this characteristic. There are also cases where there is more than one charset type header, the first is often Western European, but then there is another which is Eastern European. If this could be detected, it might be a usefuil filtering approach. I also notice that there is a difference in one of the header lines: <pre>IF HEADER "Content-Type" MATCHES "/b*charset*koi8-*" WEIGHT 101 IF HEADER "Content-Type" MATCHES "/b*windows-1251*" WEIGHT 101 i.e. no charset for the windows-1251 line.</pre> I must admit that some of the content control rules confuse me. I am not sure how useful the /b is and, maybe without it, the headers with the newline and tab in them would be caught. Gordon

tBB

posted Aug 13 '09 at 7:19 pm

[quote user="GordonM"]

Nico - Adding the rulles that you suggested seems to make a significant difference. However, there are still some "Content-Type" headers not being

detected. The charset is not on the same line as the "Content-Type" header ... there is a newline character plus a tab between them. Have you seen this as well?

[/quote]

Yes, for some reason Mercury's CC seems to be unable to detect some Content-Type headers, particularly those in a

multi-part MIME mail (therefore I've mentioned that filtering on them alone doesn't help much). I've just found that in most

cases either the subject or the sender contains the strings as well, often even mixed e.g. the Content-Type is set to

Windows-1251 and the Subject to Koi-8.

[quote user="GordonM"]

There are also cases where there is more than one charset type header, the first is often Western European, but then there
is another which is Eastern European. If this could be detected, it might be a usefuil filtering approach.

[/quote] 
I think I haven't seen these yet. Could you give me a example?


[quote user="GordonM"]
I must admit that some of the content control rules confuse me.  I am not sure how useful the /b is and, maybe without it, 
the headers with the newline and tab in them would be caught.

[/quote]

/b matches a start-of-word boundary (including start-of-line) but if I remember right it doesn't make much of a difference in this case.

I guess that's just a remnant from my tests about why the Content-Type header doesn't match in some cases and as the /b doesn't

disturb I've left it :)

Best regards,

Nico

Edit: For some reason I can't remove the strange formatting above

[quote user="GordonM"]Nico - Adding the rulles that you suggested seems to make a significant difference.  However, there are still some "Content-Type" headers not beingdetected. The charset is not on the same line as the "Content-Type" header ... there is a newline character plus a tab between them. Have you seen this as well? [/quote]  Yes, for some reason Mercury's CC seems to be unable to detect some Content-Type headers, particularly those in amulti-part MIME mail (therefore I've mentioned that filtering on them alone doesn't help much). I've just found that in mostcases either the subject or the sender contains the strings as well, often even mixed e.g. the Content-Type is set toWindows-1251 and the Subject to Koi-8. <PRE>[quote user="GordonM"] </PRE><PRE>There are also cases where there is more than one charset type header, the first is often Western European, but then there is another which is Eastern European. If this could be detected, it might be a usefuil filtering approach. [/quote] I think I haven't seen these yet. Could you give me a example? </PRE><PRE><PRE> [quote user="GordonM"]</PRE>I must admit that some of the content control rules confuse me. I am not sure how useful the /b is and, maybe without it, the headers with the newline and tab in them would be caught.</PRE><PRE>[/quote] </PRE>/b matches a start-of-word boundary (including start-of-line) but if I remember right it doesn't make much of a difference in this case. I guess that's just a remnant from my tests about why the Content-Type header doesn't match in some cases and as the /b doesn'tdisturb I've left it :)Best regards,NicoEdit: For some reason I can't remove the strange formatting above

GordonM

posted Aug 15 '09 at 3:31 pm

I wonder if someone could provide a definitive answer as to whether Content Control will consider:

Content-Type: text/plain;
charset="koi8-r"

as all part of the Content-Type header (given the newline, followed by a tab character).

If not, is there any way of detecting the line containing charset?

Thank you

Gordon

I wonder if someone could provide a definitive answer as to whether Content Control will consider: Content-Type: text/plain; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; charset="koi8-r" as all part of the Content-Type header (given the newline, followed by a tab character). If not, is there any way of detecting the line containing charset? Thank you Gordon

Rolf Lindby

posted Aug 15 '09 at 4:19 pm

Header folding is allowed according to the rules in RFC 822, and Mercury should unfold any such header before evaluating it. I can't say why this appeared to fail in the case you mention without seeing the message, but one explanation could be that it was a sub-part header, not a message header.

/Rolf

Header folding is allowed according to the rules in RFC 822, and Mercury should unfold any such header before evaluating it. I can't say why this appeared to fail in the case you mention without seeing the message, but one explanation could be that it was a sub-part header, not a message header./Rolf&nbsp;

GordonM

posted Aug 15 '09 at 4:56 pm

Rolf - Thank you for the useful reply. The following is one example that fauiled:

X-Apparently-To: gordon DELETED Sat, 15 Aug 2009 04:46:43 -0700
X-YahooFilteredBulk: 156.17.227.111
X-YMailISG: 2Wx3OGIWLDtCzubD43rje1eW5SiVb6Fl2Ry9CdiN8LAvp7k4Xk2dLVAY2W4RjBAaNLtyuZhmrfQ5YS6LDnZumcYWZF5eCm3VLdDdJ0mW17S8pLNRx1nS7yljBr_jM3c9fsi7BNCExDPVx8seC.l1sgocMiySJ9qPU1BLqhZu0LjHF3K93oG5aukYNpJR0xuPYaIQHmdJmVCQnzP.UGEVRWJcXCJyOuVKONiHtSwmkeBzSwSdBW6fjVtbsBwk_1w6XKeQ.0OYM_rxxs7wPlPcp.WF0L.PRUpUPlbR5m.VJngRiKuZ_PRa9pi2tao3ZDoOBFVHR3mur4Dk
X-Originating-IP: [156.17.227.111]
Authentication-Results: mta103.rog.mail.sp1.yahoo.com  from=blackwell.demon.nl; domainkeys=neutral (no sig); from=blackwell.demon.nl; dkim=neutral (no  sig)
Received: from 156.17.227.111  (HELO be99gz2) (156.17.227.111)
  by mta103.rog.mail.sp1.yahoo.com with SMTP; Sat, 15 Aug 2009 04:46:43 -0700
Message-ID: <000701ca1d9e$1a426e80$4a37416a@blackwell.demon.nl>
Reply-To: "Milagros McFarland" <milagros_mcFarland_xf@blackwell.demon.nl>
From: "Milagros McFarland" <milagros_mcFarland_xf@blackwell.demon.nl>
To: <goneill@rogers.com>, 
	<DELETED, 
	<goubar@rogers.com>
Subject: [Bulk] No Doctor needed to order online!
Date: Sat, 15 Aug 2009 06:47:01 -0500
MIME-Version: 1.0
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Office Outlook, Build 11.0.5510
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1807
X-Antivirus: AVG for E-mail 8.5.392 [270.13.57/2303]
Content-Type: multipart/mixed; boundary="=======AVGMAIL-0A162BE6======="
--=======AVGMAIL-0A162BE6=======
Content-Type: text/plain;
	format=flowed;
	charset="windows-1250"
	reply-type=original
Content-Transfer-Encoding: 7bit
0rder_your Favorite RxMedications 0nline!..N0_Doctor_Needed!

As you can see, there is a sub? part preceding the charset directive. Is charset="windows-1250", therefore, considered to be part of a sub=-part? If so, does this mean that Content Control has no way of recognizing it?

Thank you

Gordon

P.S. I have noticed some oddities about the forum message entry windows. Sometimes word-wrap stops happening and, when I enter a new line, I always get spacing as if it were two newlines. I am not sure whether this is a local problem for me or just a feature of the forum software.

Rolf - Thank you for the useful reply.&nbsp; The following is one example that fauiled:<PRE id=line1>X-Apparently-To: gordon DELETED Sat, 15 Aug 2009 04:46:43 -0700 X-YahooFilteredBulk: 156.17.227.111 X-YMailISG: 2Wx3OGIWLDtCzubD43rje1eW5SiVb6Fl2Ry9CdiN8LAvp7k4Xk2dLVAY2W4RjBAaNLtyuZhmrfQ5YS6LDnZumcYWZF5eCm3VLdDdJ0mW17S8pLNRx1nS7yljBr_jM3c9fsi7BNCExDPVx8seC.l1sgocMiySJ9qPU1BLqhZu0LjHF3K93oG5aukYNpJR0xuPYaIQHmdJmVCQnzP.UGEVRWJcXCJyOuVKONiHtSwmkeBzSwSdBW6fjVtbsBwk_1w6XKeQ.0OYM_rxxs7wPlPcp.WF0L.PRUpUPlbR5m.VJngRiKuZ_PRa9pi2tao3ZDoOBFVHR3mur4Dk X-Originating-IP: [156.17.227.111] Authentication-Results: mta103.rog.mail.sp1.yahoo.com from=blackwell.demon.nl; domainkeys=neutral (no sig); from=blackwell.demon.nl; dkim=neutral (no sig) Received: from 156.17.227.111 (HELO be99gz2) (156.17.227.111) by mta103.rog.mail.sp1.yahoo.com with SMTP; Sat, 15 Aug 2009 04:46:43 -0700 Message-ID: &lt;000701ca1d9e$1a426e80$4a37416a@blackwell.demon.nl&gt; Reply-To: "Milagros McFarland" &lt;milagros_mcFarland_xf@blackwell.demon.nl&gt; From: "Milagros McFarland" &lt;milagros_mcFarland_xf@blackwell.demon.nl&gt; To: &lt;goneill@rogers.com&gt;, &lt;DELETED, &lt;goubar@rogers.com&gt; Subject: [Bulk] No Doctor needed to order online! Date: Sat, 15 Aug 2009 06:47:01 -0500 MIME-Version: 1.0 X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Office Outlook, Build 11.0.5510 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1807 X-Antivirus: AVG for E-mail 8.5.392 [270.13.57/2303] Content-Type: multipart/mixed; boundary="=======AVGMAIL-0A162BE6=======" --=======AVGMAIL-0A162BE6======= Content-Type: text/plain; format=flowed; charset="windows-1250" reply-type=original Content-Transfer-Encoding: 7bit 0rder_your Favorite RxMedications 0nline!..N0_Doctor_Needed!</PRE> As you can see, there is a sub? part preceding the charset directive.&nbsp; Is charset="windows-1250", therefore, considered to be part of a sub=-part?&nbsp; If so, does this mean that Content Control has no way of recognizing it? Thank you Gordon P.S. I have noticed some oddities about the forum message entry windows.&nbsp; Sometimes word-wrap stops happening and, when I enter a new line, I always get spacing as if it were two newlines.&nbsp; I am not sure whether this is a local problem for me or just a feature of the forum software. &nbsp;

Thomas R. Stephenson

posted Aug 15 '09 at 6:09 pm

If not, is there any way of detecting the line containing charset?

The is always a way to catch any string, you do the matching on the expression in the message.

<blockquote>If not, is there any way of detecting the line containing charset?</blockquote>The is always a way to catch any string, you do the matching on the expression in the message.

Rolf Lindby

posted Aug 15 '09 at 6:38 pm

If you test against header Content-Type in that message it will be multipart/mixed; boundary="=======AVGMAIL-0A162BE6=======" that's evaluated. charset="windows-1250" is a sub-part header.

The HTML formatting in the forum usually separates paragraphs with labels, which will give some extra space inbetween. If you get strange results because of this you have the option to edit the HTML code by clicking the HTML tool button. For those not so familiar with HTML that's probably not a very practical solution, though.

/Rolf

If you test against header Content-Type in that message it will be&nbsp;multipart/mixed; boundary="=======AVGMAIL-0A162BE6=======" that's evaluated.&nbsp;charset="windows-1250" is a sub-part header.The HTML formatting in the forum usually separates paragraphs with &lt;p&gt; labels, which will give some extra space inbetween. If you get strange results because of this you have the option to edit the HTML code by clicking the HTML tool button. For those not so familiar with HTML that's probably not a very practical solution, though./Rolf&nbsp;

GordonM

posted Aug 15 '09 at 8:10 pm

Thomas - Does that mean that I should check the body of the message for charset etc, even though the content type information doesn't visibly appear in the body when viewed normally with an e-mail client?

Thank you

Gordon

Thomas&nbsp;- Does that mean that I should check the body of the message for charset etc, even though the content type information doesn't visibly appear in the body when viewed normally with an e-mail client? Thank you Gordon

Thomas R. Stephenson

posted Aug 16 '09 at 6:28 pm

Thomas - Does that mean that I should check the body of the message for charset etc, even though the content type information doesn't visibly appear in the body when viewed normally with an e-mail client?

Correct, just like any other string.

<blockquote>Thomas&nbsp;- Does that mean that I should check the body of the message for charset etc, even though the content type information doesn't visibly appear in the body when viewed normally with an e-mail client?</blockquote>Correct, just like any other string.&nbsp;

GordonM

posted Aug 16 '09 at 7:20 pm

Well, maybe I have the definitions wrong. I am using:

#

Body or sub-part

IF BODY MATCHES

"*charset=?iso-2022-jp*" WEIGHT 112 tag "foreign"

IF BODY MATCHES

"*charset=?koi8-r*" WEIGHT 112 tag "foreign"

IF BODY MATCHES

"*charset=?windows-1251*" WEIGHT 112 tag "foreign"

IF BODY MATCHES

"*charset=?Big5*" WEIGHT 112 tag "foreign"

IF BODY MATCHES

"*charset=?GB2312*" WEIGHT 112 tag "foreign"

IF BODY MATCHES

"*charset=?GB18030*" WEIGHT 112 tag "foreign"

IF BODY MATCHES

"*charset=?iso-8859-2*" WEIGHT 112 tag "foreign"

IF BODY MATCHES

"*charset=?windows-1250*" WEIGHT 112 tag "foreign"

#

Body or sub-part ALTERNATIVE

IF BODY MATCHES

"*charset=\"iso-2022-jp\"*" WEIGHT 111 tag "foreign"

IF BODY MATCHES

"*charset=\"koi8-r\"*" WEIGHT 111 tag "foreign"

IF BODY MATCHES

"*charset=\"windows-1251\"*" WEIGHT 111 tag "foreign"

IF BODY MATCHES

"*charset=\"Big5\"*" WEIGHT 111 tag "foreign"

IF BODY MATCHES

"*charset=\"B2312\"*" WEIGHT 111 tag "foreign"

IF BODY MATCHES

"*charset=\"GB18030\"*" WEIGHT 111 tag "foreign"

and none of these seems to catch anything. the "?" characters are to account for the leading double quote that, I presume, has to be considered or has to be escaped, as in the first set of rules (I am using the different weights to recognize what is doing what).

Gordon

Well, maybe I have the definitions wrong.&nbsp; I am using: #Body or sub-part IF BODY MATCHES "*charset=?iso-2022-jp*" WEIGHT 112 tag "foreign" IF BODY MATCHES "*charset=?koi8-r*" WEIGHT 112 tag "foreign" IF BODY MATCHES "*charset=?windows-1251*" WEIGHT 112 tag "foreign" IF BODY MATCHES "*charset=?Big5*" WEIGHT 112 tag "foreign" IF BODY MATCHES "*charset=?GB2312*" WEIGHT 112 tag "foreign" IF BODY MATCHES "*charset=?GB18030*" WEIGHT 112 tag "foreign" IF BODY MATCHES "*charset=?iso-8859-2*" WEIGHT 112 tag "foreign" IF BODY MATCHES "*charset=?windows-1250*" WEIGHT 112 tag "foreign" # #Body or sub-part ALTERNATIVE IF BODY MATCHES "*charset=\"iso-2022-jp\"*" WEIGHT 111 tag "foreign" IF BODY MATCHES "*charset=\"koi8-r\"*" WEIGHT 111 tag "foreign" IF BODY MATCHES "*charset=\"windows-1251\"*" WEIGHT 111 tag "foreign" IF BODY MATCHES "*charset=\"Big5\"*" WEIGHT 111 tag "foreign" IF BODY MATCHES "*charset=\"B2312\"*" WEIGHT 111 tag "foreign" IF BODY MATCHES "*charset=\"GB18030\"*" WEIGHT 111 tag "foreign" and none of these seems to catch anything.&nbsp;&nbsp;the "?"&nbsp;characters are to account for the leading double quote that, I presume, has&nbsp;to be considered or has to be escaped, as in the first set of rules (I am using the different weights to recognize what is doing what). Gordon &nbsp; &nbsp;

Thomas R. Stephenson

posted Aug 17 '09 at 7:24 am

I would have used something like if body hasall "charset=, koi8-r" weight 122 Might get a few false positive and that's why I'm only using POPFileD any more to catch spam. I simply do want to spend the time trying to raise my effectiveness from 99.87% that last 0.13%. ;-)

I would have used something like if body hasall "charset=, koi8-r" weight 122&nbsp; Might get a few false positive and that's why I'm only using POPFileD any more to catch spam.&nbsp; I simply do want to spend the time trying to raise my effectiveness from 99.87% that last 0.13%.&nbsp; ;-)

GordonM

posted Aug 17 '09 at 4:21 pm

Thomas - Unfortunately, your "HASALL" suggestion doesn't catch the charset definitions either. Here are the rules that I have put in place:

# Body or sub-part SECOND ALTERNATIVE

IF BODY HASALL "charset=, iso-2022-jp" WEIGHT 112 tag "foreign"

IF BODY HASALL "charset=, koi8-r" WEIGHT 112 tag "foreign"

IF BODY HASALL "charset=, windows-1251" WEIGHT 112 tag "foreign"

IF BODY HASALL "charset=, Big5" WEIGHT 112 tag "foreign"

IF BODY HASALL "charset=, B2312" WEIGHT 112 tag "foreign"

IF BODY HASALL "charset=, GB18030" WEIGHT 112 tag "foreign"

IF BODY HASALL "charset=, iso-8859-2" WEIGHT 112 tag "foreign"

IF BODY HASALL "charset=, windows-1250" WEIGHT 112 tag "foreign"

The following extract from some SPAM source was not caught, which I think should have been:

Subject: =?koi8-r?B?8NLJx8zB28HFzSDOwSDtxdbE1c7B0s/EztnKIMHXycHDyc/Ozs8tyw==?=	=?koi8-r?B?z9PNyd7F08vJyiDTwczPziA8PO3h6/MtMjAwOT4+Lg==?=
Date: Mon, 17 Aug 2009 13:50:16 +0100
MIME-Version: 1.0
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 5.00.2314.1300
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300
X-Antivirus: AVG for E-mail 8.5.392 [270.13.58/2308]
Content-Type: multipart/mixed; boundary="=======AVGMAIL-36CF6827======="
X-UC-Weight: [### ] 407
X-CC-Diagnostic: foreign (51), foreign (51), foreign (51), foreign (51),
Header To contains "hibou@" (51), Sender matches "*.ru>*" (51)
--=======AVGMAIL-36CF6827=======
Content-Type: text/plain;
	charset="koi8-r"
Content-Transfer-Encoding: 8bit
В период c 22 по 23 августа 2009 года в подмосковном наукограде Жуковский

I would appreciate any further ideas.

Thank you

Gordon

Thomas - Unfortunately, your "HASALL" suggestion doesn't catch the charset definitions either.&nbsp; Here are the rules that I have put in place: <BLOCKQUOTE> &nbsp;# Body or sub-part SECOND ALTERNATIVE IF BODY HASALL "charset=, iso-2022-jp" WEIGHT 112 tag "foreign" IF BODY HASALL "charset=, koi8-r" WEIGHT 112 tag "foreign" IF BODY HASALL "charset=, windows-1251" WEIGHT 112 tag "foreign" IF BODY HASALL "charset=, Big5" WEIGHT 112 tag "foreign" IF BODY HASALL "charset=, B2312" WEIGHT 112 tag "foreign" IF BODY HASALL "charset=, GB18030" WEIGHT 112 tag "foreign" IF BODY HASALL "charset=, iso-8859-2" WEIGHT 112 tag "foreign" IF BODY HASALL "charset=, windows-1250" WEIGHT 112 tag "foreign"</BLOCKQUOTE> &nbsp;The following extract from some SPAM source was not caught, which I think should have been: <BLOCKQUOTE><PRE id=line1>Subject: =?koi8-r?B?8NLJx8zB28HFzSDOwSDtxdbE1c7B0s/EztnKIMHXycHDyc/Ozs8tyw==?= =?koi8-r?B?z9PNyd7F08vJyiDTwczPziA8PO3h6/MtMjAwOT4+Lg==?= Date: Mon, 17 Aug 2009 13:50:16 +0100 MIME-Version: 1.0 X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.2314.1300 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300 X-Antivirus: AVG for E-mail 8.5.392 [270.13.58/2308] Content-Type: multipart/mixed; boundary="=======AVGMAIL-36CF6827=======" X-UC-Weight: [### ] 407 X-CC-Diagnostic: foreign (51), foreign (51), foreign (51), foreign (51), Header To contains "hibou@" (51), Sender matches "*.ru&gt;*" (51) --=======AVGMAIL-36CF6827======= Content-Type: text/plain; charset="koi8-r" Content-Transfer-Encoding: 8bit В период c 22 по 23 августа 2009 года в подмосковном наукограде Жуковский</PRE></BLOCKQUOTE> I would appreciate any further ideas. Thank you Gordon

PaulW

posted Aug 17 '09 at 4:33 pm

As someone has already said, you cannot use Content Control to catch the charset in MIME parts. Only filtering will look at those parts of the email body.

As someone has already said, you cannot use Content Control to catch the charset in MIME parts.&nbsp; Only&nbsp;filtering will look at those parts of the email body. &nbsp; &nbsp;

tBB

posted Aug 17 '09 at 4:48 pm

Gordon, you can bet on it that I've tried long enough before stating in the public that Mercury is unable to detect something. Besides, the subject of the mail you've posted contains "koi8-r" and thus shoud have been well caught by the rules I've posted above.

Best regards,

Nico

Gordon, you can bet on it that I've tried long enough before stating in the public that Mercury is unable to detect something. Besides, the subject of the mail you've posted contains "koi8-r" and thus shoud have been well caught by the rules I've posted above.Best regards,Nico

Related Topics

Pending draft

Confirm move posts

Insufficient permissions

Select a different topic

Edit history