Home   Help Search Groups Login Register  
You are not logged in. To get the full experience of these forums, we recommend you log in or register
Plusnet Usergroup » All Users - The Open Forum » Plusnet Network and Technical Issues » Training spam filter - any point?
Pages: 1 [2] 3 4 ... 16
  Print  
Author Topic: Training spam filter - any point?  (Read 167997 times)
dusty_bin

Posts: 84

« Reply #15 on: May 31, 2007, 11:44:06 am »

As it is, with a Webmail client (When we have locked down what we want) a "Report Spam" button will be a must have.
Yes! and also a "Not Spam" button for false positives in the spam folder(s).
If you follow the Pegasus Mail approach, you could also do it automatically when email is manually moved into or out of the spam folder(s)?
Quote
From people's own email software though, there is only so much we can do as we need to see and verify the emails before we train the spam filter.
Indeed...
When forwarding emails using Pegasus Mail, one option is "Forward the messages without editing (Redirect, or "bounce)" which looks to be just the ticket Wink
cogilvie

Posts: 798


« Reply #16 on: May 31, 2007, 11:47:56 am »

When forwarding emails using Pegasus Mail, one option is "Forward the messages without editing (Redirect, or "bounce)" which looks to be just the ticket Wink

I suspect you'll lose the original headers at a guess, as it would be sending through your SMTP server etc. so you'll still be in the same state as you were.

Colin Ogilvie
Application Developer
Plusnet
jelv1

Posts: 2130

« Reply #17 on: May 31, 2007, 02:35:50 pm »

The advise we have had from Bob is that the full headers are needed. If you just forward the message without pasting in the headers it is apparently no use and you are just wasting your time.

jelv
dusty_bin

Posts: 84

« Reply #18 on: May 31, 2007, 04:31:03 pm »

Good points, but I just carried out an experiment and got the following headers from a forwarded message:

Received: from host2.somewhereelse.net by myinternalserver with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13)
   id LS1Y8T9Q; Thu, 31 May 2007 16:08:31 +0100
Received: from username.plus.com (81-174-211-110.pth-as4.dial.plus.net
   [81.174.211.110])by host1.somewhereelse.net (8.9.3 (PHNE_29774)/8.9.3)
   with SMTP id QAA21776for <me@username.plus.com>; Thu, 31 May 2007
   16:03:39 +0100 (BST)
Resent-Message-Id: <200705311503.QAA21776@somewhereelse.net>
Resent-from: <me@username.plus.com>
Resent-to: me@somewhereelse.net
Resent-date: Thu, 31 May 2007 16:03:25 +0100
Received: from mail.plus.net by username.plus.com with POP3
   (Mailtraq/2.2.0.1340)         id MYBL57F41F14; Mon, 26 Mar 2007 12:00:57
   +0100
Envelope-to: me@username.plus.com
Delivery-date: Mon, 26 Mar 2007 08:29:16 +0000
Received: from c2bthomr12.btconnect.com ([213.123.20.144])  by
   pih-sunmxcore10.plus.net with esmtp (PlusNet MXCore v2.00) id
   1HVkZn-0006cP-UF   for me@username.plus.com; Mon, 26 Mar 2007 08:29:15
   +0000
Received: from somebody by c2bthomr12.btconnect.comwith ESMTP id
   CPA74008;Mon, 26 Mar 2007 09:30:34 +0100 (BST)
Message-ID: <000901c76f81$fa86efc0$1402a8c0@somebody>


I hope it's not too munged to understand...  The orignal email was collected by POP3 by a server and then passed to Pesagus Mail, but I don't think it would be different if email were directly collected - if I get a chance, I may try that as well.

I suppose the question is whether the PN system would pick out the right headers.
bpullen
Plusnet Staff

Posts: 1980


WWW
« Reply #19 on: May 31, 2007, 04:37:39 pm »

Spam filtering is based on many things, some are in the body (HTML usage, misformed mime etc) some are in the headers (mail has passed through a host without rDNS, invalid headers).

The more the better basically.

cogilvie

Posts: 798


« Reply #20 on: May 31, 2007, 04:56:49 pm »

Good points, but I just carried out an experiment and got the following headers from a forwarded message:

[snip]

I hope it's not too munged to understand...  The orignal email was collected by POP3 by a server and then passed to Pesagus Mail, but I don't think it would be different if email were directly collected - if I get a chance, I may try that as well.

I suppose the question is whether the PN system would pick out the right headers.

Ah, so it's slightly more intelligent than I was giving it credit for smiley

Colin Ogilvie
Application Developer
Plusnet
dusty_bin

Posts: 84

« Reply #21 on: May 31, 2007, 06:05:07 pm »

Ah, so it's slightly more intelligent than I was giving it credit for smiley

And it did the same with a direct connection to the PN servers:
Code:
Envelope-to: test@username1.plus.com
Delivery-date: Thu, 31 May 2007 16:47:15 +0000
Received: by pih-sunmxcore16.plus.net with spam-scanned (PlusNet MXCore v2.00) id 1Htnnu-000712-UP
     for test@username1.plus.com; Thu, 31 May 2007 16:47:15 +0000
X-Daemon-Classification: INNOCENT
Received: from ptb-relay01.plus.net ([212.159.14.212])
     by pih-sunmxcore16.plus.net with esmtp (PlusNet MXCore v2.00) id 1Htnnu-0006zl-0J
     for test@username1.plus.com; Thu, 31 May 2007 16:47:14 +0000
Received: from [81.174.210.98] (helo=[81.174.210.98])
     by ptb-relay01.plus.net with esmtp (Exim) id 1HtnoJ-0006Ch-D5
     for test@username1.plus.com; Thu, 31 May 2007 17:47:40 +0100
Resent-from: "Test" <test@username2.plus.com>
Resent-to: test@username1.plus.com
Resent-date: Thu, 31 May 2007 17:44:34 +0100
Received: from ptb-cgirelay02.plus.net ([195.166.130.41])
     by pih-sunmxcore12.plus.net with esmtp (PlusNet MXCore v2.00) id 1Htnfw-0001iV-KY
     for test@username2.plus.com; Thu, 31 May 2007 16:39:00 +0000
Received: from ge0-1-0-7.ptn-gw2.plus.net ([212.159.6.51]:42273 helo=webmail.plus.net)
     by ptb-cgirelay02.plus.net with esmtp (Exim 4.34)
     id 1Htnd0-0008Fp-Nw
     for test@username2.plus.com; Thu, 31 May 2007 17:35:58 +0100
Received: from 81.174.210.98
     (SquirrelMail authenticated user username1)
     by webmail.plus.net with HTTP;
     Thu, 31 May 2007 17:35:58 +0100 (BST)
Message-ID: <22516.81.174.210.98.1180629358.squirrel@webmail.plus.net>
Date: Thu, 31 May 2007 17:35:58 +0100 (BST)
Subject: Test
From: username1@username1.plus.com
To: test@username2.plus.com
User-Agent: SquirrelMail
MIME-Version: 1.0
Content-Type: text/plain;charset=iso-8859-1
Content-Transfer-Encoding: 8bit
X-Priority: 3 (Normal)
Importance: Normal
X-PN-Spam-Filtered: by PlusNet MXCore (v4.00)
[snip the spam stuff]
Resent-Message-Id: <E1Htnnu-000712-UP@pih-sunmxcore16.plus.net>
Resent-Date: Thu, 31 May 2007 16:47:15 +0000

Now I just need to get it's spamchecker trained...
Mark1

Posts: 4

« Reply #22 on: June 01, 2007, 12:16:19 am »

Re Pegasus Mail...
Ah, so it's slightly more intelligent than I was giving it credit for smiley
I've been using Pegasus Mail for some years, and it's a very capable mail client. It lags behind the more famous mail programs in its user interface, and lacks bells and whistles such as an entire Office system attached to every action, but it generally works very well.

Its own spam filter is very effective, once trained. It learns as one moves messages into the Junk folder (trains them as spam) or out of it (trains as not-spam).
NB
Usergroup Member

Posts: 2071

« Reply #23 on: June 01, 2007, 12:22:29 am »

I believe the mail client in Opera can do redirects straight out of the box, and thunderbird can do them with the addition of a plugin.
rsarwar

Posts: 2

« Reply #24 on: June 01, 2007, 10:04:55 am »

Forwarding headers is useful as the header contain information from DSpam, this information show why DSpam thinks of a mail as Spam. I believe when you forward a mail using 'Forward' from mail client, the headers containing hop and delivery information are forwarded however headers containing other Mime tags do not get forwarded, I have to test this today.

Any mails which contain attachments, very few text or blank body are useless to training system and they normally discarded by housekeeping team.

godsell4

Posts: 397

« Reply #25 on: June 01, 2007, 02:05:37 pm »

... very few text or blank body are useless to training system and they normally discarded by housekeeping team.

The fact is, it is quite a few of these short messages with an attached .gif that are highly offensive SPAM.

So should they really be discarded ... many of them come via known open relays on bl.spamcop.net ... going to fix that hole?

SW.

BBYW1/10GB
godsell4

Posts: 397

« Reply #26 on: June 01, 2007, 06:47:00 pm »


How did this one get through, see the headers via spamcop here

It shows the DSPAM factors as:
Quote
X-DSPAM-Result: Innocent
X-DSPAM-Processed: Fri Jun  1 15:07:14 2007
X-DSPAM-Confidence: 0.5053
X-DSPAM-Improbability: 1 in 103 chance of being spam
X-DSPAM-Probability: 0.0000
X-DSPAM-Factors: 27,
   Received*Jun, 0.00181,
   Received*HW, 0.01000,
   Date*17+00, 0.01000,
   the+improved, 0.99000,
   Date*2007, 0.99000,
   Subject*Multiple, 0.99000,
   has+ever, 0.01000,
   Received*0004Xl, 0.99000,
   best+thing, 0.01000,
   Date*Jun, 0.07580,
   to+me, 0.08830,
   Due+to, 0.09087,
   Date*Fri+1, 0.09383,
   Due, 0.12463,
   improved, 0.85701,
   Received*Fri+01, 0.14402,
   x-open-relay*is, 0.84924,
   x-open-relay*a, 0.84924,
   x-open-relay*black+list, 0.84924,
   x-open-relay*list, 0.84924,
   x-open-relay*bl.spamcop.net, 0.84924,
   x-open-relay*at, 0.84924,
   x-open-relay*at+bl.spamcop.net, 0.84924,
   x-open-relay*black, 0.84924,
   x-open-relay*in, 0.84924,
   x-open-relay*in+a, 0.84924,
   x-open-relay*a+black, 0.84924

Just how many times does DSPAM need to see the x-open-relay string to decide a message is actually SPAM?  angry

SW

BBYW1/10GB
Oldjim

Posts: 1016

« Reply #27 on: June 01, 2007, 07:38:32 pm »

Has someone got sense of humour - this is taken from a mail correctly identified as spam
Quote
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 6.00.2900.2180
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2180
X-PN-VirusFiltered: by PlusNet MXCore (v4.00)
X-DSPAM-Result: Spam
X-DSPAM-Processed: Fri Jun 1 14:41:03 2007
X-DSPAM-Confidence: 0.6963
X-DSPAM-Improbability: 1 in 230 chance of being ham
X-DSPAM-Probability: 1.0000
X-DSPAM-Factors: 15,
poppy

Posts: 142

« Reply #28 on: June 02, 2007, 11:44:18 am »

With reference to Bob's message (PUGIT 305) about setting up redirects for spam@despamchecker, please can someone  clarify exactly how this should be done?

Force 9
Joined 03 June 2004
Oldjim

Posts: 1016

« Reply #29 on: June 02, 2007, 12:16:57 pm »

I thought that just forwarding the email without adding the header manually was of no use - can that be confirmed because if I am wrong it would be much easier
Pages: 1 [2] 3 4 ... 16
  Print  
 
Jump to: