[KLUG Members] junk mail filters

[Jason Edward Durrett]

On Jan 10, 2007, at 7:16 PM, Mike Williams wrote:

> Jason Edward Durrett wrote:
>> Mike Williams wrote:
>>
>>> Jason Edward Durrett wrote:
>>>
>>>> In the past month I have noticed a increase in legitimate mail not
>>>> getting to the intended recipients because  of client side junk
>>>> filters.  I have seen this even with emails that are replies to 
>>>> emails
>>>> that other people have sent.
>>>>
>>>> Now, it could be that the excuse "It was sent to my Junk Mail so I 
>>>> did
>>>> not get it" is replacing the "I got stuck in traffic" excuse, but it
>>>> seems more widespread especially with Outlook - especially replies
>>>> getting sent to Junk.  However, I have seen it as a problem with all
>>>> client filters.
>>>> Does anyone else have this problem?  Does anyone know of a way to 
>>>> get a
>>>> notification if a message is moved to Junk?  I am fairly sure there 
>>>> is
>>>> not but it does not hurt to ask.
>>>>
>>>>
>>> I don't have a solution suggestion, but I'll take a wild guess at the
>>> cause:  filters getting confused by a technique called Bayesian
>>> Poisoning.  Bayesian Filtering is where you (the user) tell the
>>> filters what is spam and what is not, and it learns to identify it. 
>>> This trick has been successful enough that spam campaigns have 
>>> started
>>> trying to confuse it by inserting phrases that are not commonly found
>>> in advertisements (bits of poetry, literary quotes, and other strange
>>> things) into the spam email.  The more of this that you tell your
>>> Bayesian filter is spam, the more likely it will get confused and
>>> quarantine something legitimate or let something through.
>>> Filters are not perfect, and all they can do is have false positive
>>> and false negative rates that are "as low as possible".  Spam
>>> campaigns keep getting sneakier, and unless there is a major change 
>>> in
>>> the way the Internet works, it will always be a battle between those
>>> who are trying to keep it out and those who are trying to sneak it 
>>> in.
>>> _______________________________________________
>>>
>>
>> You are quite right with the problem.   There is another problem, 
>> though
>> - and that is how to make sure email continues to be a reliable form 
>> of
>> communication.  It seems to me that the filters are making email less
>> reliable - does anyone know of a client side filter that alerts the
>> sender that the message is never going to be read?
>>
>>
> No, and there's unlikely to be one because it would be too useful to 
> spammers.  They would get instant feedback on whether or not their 
> most recent trick to beat the filters is working.
>
> Filters are our best hope at the moment for keeping email a usable 
> form of communication.  Without them, the signal to noise ratio of 
> email would be so low it would be useless.  The way the economics of 
> the Net work, this could still happen even with the filtering 
> technologies available.
> _______________________________________________
> Members mailing list
> Members at kalamazoolinux.org
> 

On  my network I use a combination of grey-listing, dbls, rdns checks, 
syntax checks, etc during smtp - this is extremely effective at 
rejecting spam while still notifying a sender in case there is a false 
positive.  I have been fooling with using spam assassin during smtp but 
have not put it into production yet (sa-exim) - when I do that should 
eliminate the small trickle of spam on my network now - about 98% of 
the connections I get are dropped and an explicit reject is sent.

Before I implemented this I was using filters, mainly on thunderbird on 
windows machines and mail on os x machines - They became useless 
because the sales people were getting orders, the orders were going to 
junk,  the customers were calling weeks later and complaining that they 
never got their shipment . . . . etc etc.  Everyone had to be reminded 
to check their junk mail daily - so the spam was actually getting 
through.

My concern is that as more people use filters as a solution, the more 
legitimate mail goes to them and the more the IT guys have to walk 
around the office reminding people to check their junk mail . . .