On Saturday, 13 February 2021 at 10:28:32 UTC, The Natural Philosopher wrote:

Given that large mail providers can't perform filtering during receipt of mail (i.e. whilst the SMTP dialogue is still underway)
They could if they wanted to

No, it's not a viable approach for a large mail platform / enterprise.

High volume mail servers are effectively just bastion hosts; their job is simply to accept mail. In some cases they might perform rudimentary checks (DNSBL lookups usually) and bounce during the SMTP dialogue but any content inspection is the role of the internal servers dedicated to the heavy lifting of that task. You can accept an email in a fraction of the time it takes to scan and filter it - the roles are entirely different and for providers likely to be attacked you would never expose machines performing the latter function to the outside world.

On 13/02/2021 09:00, Tim Streater wrote:
Given that large mail providers can't perform filtering during receipt of mail (i.e. whilst the SMTP dialogue is still underway) if a message is subsequently found to be spam ...

What d'ye mean "subsequently found to be spam"? It's not the business of an
email transporter to have an opinion about what is spam or not. That's a
matter for the recipient, whose job it is to train their email client to do
the filtering.

Email 'transporter'? The like of Hotmail, Gmail etc are much more than that - it's part of their service offering to filter mailicious and unwanted mail. Both require a judgement, but they are in a much better position than the recipient to perform that function hence why they offer it.

If you don't like the service(s) being offered then shop around to one that you do - it'd be interesting to hear if there are any mail platforms that don't perform any filtering and deliver *everything* to your Inbox though. Of course, as this is a DIY group, you can also do it yourself and hence have full control.

On 13/02/2021 12:28, Mathew Newton wrote:
On 13/02/2021 09:00, Tim Streater wrote:
Given that large mail providers can't perform filtering during receipt of mail (i.e. whilst the SMTP dialogue is still underway) if a message is subsequently found to be spam ...

What d'ye mean "subsequently found to be spam"? It's not the business of an
email transporter to have an opinion about what is spam or not. That's a
matter for the recipient, whose job it is to train their email client to do
the filtering.

Email 'transporter'? The like of Hotmail, Gmail etc are much more than that - it's part of their service offering to filter mailicious and unwanted mail. Both require a judgement, but they are in a much better position than the recipient to perform that function hence why they offer it.

If you don't like the service(s) being offered then shop around to one that you do - it'd be interesting to hear if there are any mail platforms that don't perform any filtering and deliver *everything* to your Inbox though. Of course, as this is a DIY group, you can also do it yourself and hence have full control.

which I do


--
€œThe fundamental cause of the trouble in the modern world today is that
the stupid are cocksure while the intelligent are full of doubt."

- Bertrand Russell

Mathew Newton wrote:
On 13/02/2021 09:00, Tim Streater wrote:
Given that large mail providers can't perform filtering during receipt
of mail (i.e. whilst the SMTP dialogue is still underway) if a message
is subsequently found to be spam ...

What d'ye mean "subsequently found to be spam"? It's not the business of an
email transporter to have an opinion about what is spam or not. That's a
matter for the recipient, whose job it is to train their email client to do
the filtering.

Email 'transporter'? The like of Hotmail, Gmail etc are much more than
that - it's part of their service offering to filter mailicious and unwanted
mail. Both require a judgement, but they are in a much better position
than the recipient to perform that function hence why they offer it.

If you don't like the service(s) being offered then shop around to one
that you do - it'd be interesting to hear if there are any mail platforms
that don't perform any filtering and deliver *everything* to your Inbox
though. Of course, as this is a DIY group, you can also do it yourself
and hence have full control.

My hosting service gives me the choice. I have several domains hosted
there as well as web hosting. My main mail addresses there have basic
spam filtering and are forwarded (no storage) directly to a mail
server here at home. In addition though, if I create a mailbox (or
forwarder) for catchall@my domain then *all* E-mail is delivered
with no filtering. I use this to check for things that *might* be
for me by running a script that searches through the E-Mails in the
catchall box for ones that have strings such as 'chris', 'cgreen',
etc. in the To: header. Anything that might be for me I send on to my
mail server, the rest I delete.

--
Chris Green
·

In message , The Natural Philosopher
writes
On 12/02/2021 20:49, Tim Lamb wrote:
This was a *reply* to an incoming mail to which I added a PDF attachment.

PDF attachment? Obviously seriously pervy spam!

Or possibly simply TOO BIG

305kB sketch.

--
Tim Lamb

On 13/02/2021 10:28, The Natural Philosopher wrote:
On 12/02/2021 22:18, Mathew Newton wrote:
On Friday, 12 February 2021 at 14:32:41 UTC, The Natural Philosopher
wrote:

Dropping mail silently contravenes the protocol. But it is exactly what
big mail servers and spam filters do

How would you expect/prefer them to behave?

Bounce, or inform recipient

Trouble is the spam will be forged sender so you end up creating
unwanted backscatter. I drop all such bounce messages that do not
originate from one of my domains on the floor. There are a lot less now
than in the past so most big setups most be adopting a rule along the
lines of if it is obvious bulk UCE spam accept and then junk it.

The old Demon philosophy of accept everything and leave it to the end
user to sort it out is long gone now. Even Demon was forced to add
antispam measures to their email service after the great Swenfest.

Given that large mail providers can't perform filtering during receipt
of mail (i.e. whilst the SMTP dialogue is still underway)
They could if they wanted to

if a message is subsequently found to be spam it can't be sent back to
the purported sender as this is likely to fake and quarantining
everything doesn't really work for anyone (mail platform or recipient)
given the volumes involved.

They tend to score it as it is on its way to the recipients mailbox.
Increasingly email services no longer offer a catchall mailbox and you
have to define aliases for anything you want to be accepted.

In days of yore I used to get a lot of spam to partial Turnpike msgids
beginning Ewok & seq and before that Snews client msgids snz123456

I run a mail server and silently drop (as in don't deliver to the
recipient or bounce) the 'spammiest of spam' and have never knowingly
had a single false positive. I would expect large mail platforms to
perform just as well. False positives for 'possibly spam' are few and
far between these days too (and usually down to the poorly configured
sender domains/software).



--
Regards,
Martin Brown

Tim Lamb wrote:

to which I added a PDF attachment. === ding, ding, ding

There's your problem.

Some of the popular ****ty email services, they will
not pass any kind of attachment at all. It's unknown
just exactly what kind of encoding or object would
pass inspection.

They definitely don't pass EXE.

And what they're doing is not "AV scanning". That's what
they should be doing. But instead, they just reject
materials by type, out of hand. On a whim.

Try attaching a document the same byte size as your PDF,
but of type "plain.txt", without any fixed format encoding
inside such as BASE64 or S-Record or YENC. You could try
sending these items, to yourself. If they sense an attempt
to encode, inside the "plain.txt" file, it will again be
rejected. As you might be trying to hide an EXE inside
a .txt file, by encoding it.

I bet the email without the attachment, got through.

Paul

"Tim Streater" wrote in message
...
On 12 Feb 2021 at 22:16:18 GMT, "Fred" wrote:



"Tim Streater" wrote in message
...
On 12 Feb 2021 at 18:47:59 GMT, "Fred" wrote:



"The Natural Philosopher" wrote in message
...
On 12/02/2021 11:21, Jethro_uk wrote:
On Fri, 12 Feb 2021 10:29:49 +0000, The Natural Philosopher wrote:

On 12/02/2021 08:50, Tim Lamb wrote:
[quoted text muted]
A brief education on email transport.

Start with it being inherently unreliable

Actually it isn't.

Its the anti-spam that has made it so

gmail's anti spam is surprisingly reliable.

Never had it claim any email was spam when it wasn't

I prefer making the deicsion on my own machine, not having someone else
deciding for me what is and what is not spam.

I find it more convenient to have the system
do it when its as reliable as the gmail one is.

You mean because it just so happens (and with you having no control over
the
matter) that gmail makes the same decisions that you would, about what is
spam.

Nope, its obvious what is spam.

"Tim Streater" wrote in message
...
On 12 Feb 2021 at 22:18:26 GMT, Mathew Newton
wrote:

On Friday, 12 February 2021 at 14:32:41 UTC, The Natural Philosopher
wrote:

Dropping mail silently contravenes the protocol. But it is exactly what
big mail servers and spam filters do

How would you expect/prefer them to behave?

Given that large mail providers can't perform filtering during receipt of
mail (i.e. whilst the SMTP dialogue is still underway) if a message is
subsequently found to be spam ...

What d'ye mean "subsequently found to be spam"? It's not the business of
an email transporter to have an opinion about what is spam or not.

Thats very arguable indeed with a major operation
like gmail can see that the same email was sent to
a wide variety of recipients and isnt a newsletter etc.

That's a matter for the recipient, whose job it is to train their email
client to do the filtering.

Or they have enough of a clue to realise that operations
like gmail have far more data to use to decide what is
spam so they only have to do some minor training after
the bulk of it has been done by their email provider.

On Sun, 14 Feb 2021 05:36:50 +1100, Fred, better known as cantankerous
trolling senile geezer Rodent Speed, wrote:

FLUSH the trolling senile asshole's latest troll**** unread

On Sun, 14 Feb 2021 05:31:25 +1100, Fred, better known as cantankerous
trolling senile geezer Rodent Speed, wrote:


You mean because it just so happens (and with you having no control over
the
matter) that gmail makes the same decisions that you would, about what is
spam.

Nope, its obvious what is spam.

Nope, what is obvious is that you are trolling senile swine, senile Rodent!

--
The Natural Philosopher about senile Rodent:
"Rod speed is not a Brexiteer. He is an Australian troll and arsehole."
Message-ID:

"Chris Green" wrote in message
...
Fred wrote:


"Tim Streater" wrote in message
...
On 12 Feb 2021 at 18:47:59 GMT, "Fred" wrote:



"The Natural Philosopher" wrote in message
...
On 12/02/2021 11:21, Jethro_uk wrote:
On Fri, 12 Feb 2021 10:29:49 +0000, The Natural Philosopher wrote:

On 12/02/2021 08:50, Tim Lamb wrote:
[quoted text muted]
A brief education on email transport.

Start with it being inherently unreliable

Actually it isn't.

Its the anti-spam that has made it so

gmail's anti spam is surprisingly reliable.

Never had it claim any email was spam when it wasn't

I prefer making the deicsion on my own machine, not having someone else
deciding for me what is and what is not spam.

I find it more convenient to have the system
do it when its as reliable as the gmail one is.

I have unfiltered mail delivered to me from my hosting service, I
simply reject anything not specifically to me (I do include stuff that
*might* be for me by allowing To: cgreen, To: cris, etc.) and then I
just filter it myself, by looking at it.

Much more convenient to have gmail do it completely
automatically and very reliably indeed.

I do get some junk but probably only 10% of the fairly large amount of
mail I handle so it's just not an issue. An efficient, text mode,
mail program helps, huge complicated graphical mails don't waste time
getting displayed, if I open it I see the text content but that's all.

I much prefer to see stuff properly organised, particularly with bills etc.

I can decide to see the pictures if I want but that's pretty
rare unless a friend has actually sent me some pictures.

I get lots, particularly with offers of books from amazon,
quite a few of which I choose to buy or see if I can torrent.

All I have to do to get rid of an unwanted mail is hit 'd'
and it's gone, so deleting junk is pretty instantaneous.

I find it much more convenient to have that done
completely automatically and very reliably indeed.

Fred wrote:


"Chris Green" wrote in message
...
Fred wrote:


"Tim Streater" wrote in message
...
On 12 Feb 2021 at 18:47:59 GMT, "Fred" wrote:



"The Natural Philosopher" wrote in message
...
On 12/02/2021 11:21, Jethro_uk wrote:
On Fri, 12 Feb 2021 10:29:49 +0000, The Natural Philosopher wrote:

On 12/02/2021 08:50, Tim Lamb wrote:
[quoted text muted]
A brief education on email transport.

Start with it being inherently unreliable

Actually it isn't.

Its the anti-spam that has made it so

gmail's anti spam is surprisingly reliable.

Never had it claim any email was spam when it wasn't

I prefer making the deicsion on my own machine, not having someone else
deciding for me what is and what is not spam.

I find it more convenient to have the system
do it when its as reliable as the gmail one is.

I have unfiltered mail delivered to me from my hosting service, I
simply reject anything not specifically to me (I do include stuff that
*might* be for me by allowing To: cgreen, To: cris, etc.) and then I
just filter it myself, by looking at it.

Much more convenient to have gmail do it completely
automatically and very reliably indeed.

Why?

I do get some junk but probably only 10% of the fairly large amount of
mail I handle so it's just not an issue. An efficient, text mode,
mail program helps, huge complicated graphical mails don't waste time
getting displayed, if I open it I see the text content but that's all.

I much prefer to see stuff properly organised, particularly with bills etc.

Mine is, very organised by a filter program that delivers list mails
each to their own directory.


I find it much more convenient to have that done
completely automatically and very reliably indeed.

How do you know it's "very reliably indeed"?

--
Chris Green
·

In message , Paul
writes
Tim Lamb wrote:

to which I added a PDF attachment. === ding, ding, ding

There's your problem.

Some of the popular ****ty email services, they will
not pass any kind of attachment at all. It's unknown
just exactly what kind of encoding or object would
pass inspection.

They definitely don't pass EXE.

And what they're doing is not "AV scanning". That's what
they should be doing. But instead, they just reject
materials by type, out of hand. On a whim.

Try attaching a document the same byte size as your PDF,
but of type "plain.txt", without any fixed format encoding
inside such as BASE64 or S-Record or YENC. You could try
sending these items, to yourself. If they sense an attempt
to encode, inside the "plain.txt" file, it will again be
rejected. As you might be trying to hide an EXE inside
a .txt file, by encoding it.

I bet the email without the attachment, got through.

Hmm.. it got through when I forwarded it with some additional text.

Other mails have succeeded since.

--
Tim Lamb

"Chris Green" wrote in message
...
Fred wrote:


"Chris Green" wrote in message
...
Fred wrote:


"Tim Streater" wrote in message
...
On 12 Feb 2021 at 18:47:59 GMT, "Fred" wrote:



"The Natural Philosopher" wrote in message
...
On 12/02/2021 11:21, Jethro_uk wrote:
On Fri, 12 Feb 2021 10:29:49 +0000, The Natural Philosopher
wrote:

On 12/02/2021 08:50, Tim Lamb wrote:
[quoted text muted]
A brief education on email transport.

Start with it being inherently unreliable

Actually it isn't.

Its the anti-spam that has made it so

gmail's anti spam is surprisingly reliable.

Never had it claim any email was spam when it wasn't

I prefer making the deicsion on my own machine, not having someone
else
deciding for me what is and what is not spam.

I find it more convenient to have the system
do it when its as reliable as the gmail one is.

I have unfiltered mail delivered to me from my hosting service, I
simply reject anything not specifically to me (I do include stuff that
*might* be for me by allowing To: cgreen, To: cris, etc.) and then I
just filter it myself, by looking at it.

Much more convenient to have gmail do it completely
automatically and very reliably indeed.

Why?

Because all you have to do is use a gmail email address.

I have plenty of better things to do with my time than to
fart around working out what is spam and what isnt when
gmail does that completely automatically and very reliably.

I do get some junk but probably only 10% of the fairly large amount of
mail I handle so it's just not an issue. An efficient, text mode,
mail program helps, huge complicated graphical mails don't waste time
getting displayed, if I open it I see the text content but that's all.

I much prefer to see stuff properly organised, particularly with bills
etc.

Mine is, very organised by a filter program that delivers list mails
each to their own directory.

I meant the body of the email there,
not the organisation of different emails.

I find it much more convenient to have that done
completely automatically and very reliably indeed.

How do you know it's "very reliably indeed"?

By checking that nothing that should be
in the inbox is actually in the spam folder
and that no spam ends up in the inbox
and that nothing goes missing.

Tim Lamb wrote:
In message , Paul
writes
Tim Lamb wrote:

to which I added a PDF attachment. === ding, ding, ding

There's your problem.

Some of the popular ****ty email services, they will
not pass any kind of attachment at all. It's unknown
just exactly what kind of encoding or object would
pass inspection.

They definitely don't pass EXE.

And what they're doing is not "AV scanning". That's what
they should be doing. But instead, they just reject
materials by type, out of hand. On a whim.

Try attaching a document the same byte size as your PDF,
but of type "plain.txt", without any fixed format encoding
inside such as BASE64 or S-Record or YENC. You could try
sending these items, to yourself. If they sense an attempt
to encode, inside the "plain.txt" file, it will again be
rejected. As you might be trying to hide an EXE inside
a .txt file, by encoding it.

I bet the email without the attachment, got through.

Hmm.. it got through when I forwarded it with some additional text.

Other mails have succeeded since.


Do you suppose someone is using a body hash filter,
and blocking messages with absolutely identical body
text ?

You can do, say, CRC32 over the body text of a message,
keep a record of all CRCs used for Sent messages, then
if a user attempts to send the same body text, you assume
it's spam and filter it.

But that seems unlikely though.

AIOE has such a filter set up for USENET postings.
It's a nuisance for sure.

Paul

On Sun, 14 Feb 2021 08:36:12 +1100, Fred, better known as cantankerous
trolling senile geezer Rodent Speed, wrote:

FLUSH the trolling senile pest's latest troll**** unread

--
dennis@home to retarded trolling senile Rodent:
"sod off rod you don't have a clue about anything."
Message-ID:

On Saturday, 13 February 2021 at 22:15:09 UTC, Paul wrote:

Do you suppose someone is using a body hash filter,
and blocking messages with absolutely identical body
text ?

You can do, say, CRC32 over the body text of a message,
keep a record of all CRCs used for Sent messages, then
if a user attempts to send the same body text, you assume
it's spam and filter it.

But that seems unlikely though.

It's quite common actually - standard practice even. But not exactly the way you've described...

The problem with a CRC32 (hashing) is that, by design, the smallest of changes to the input causes a massive change to the output. This means it's trivial for a spammer to vary their messages by just a single byte and it'll result in a different checksums and thus render the mechanism useless. What is done instead - by the likes of Pyzor (https://www.pyzor.org/en/latest/) which is arguably the most common implementation - is that various bits of the message are removed and then a digest (similar to a hash but not quite so rigid and collision free) is created from what's left. This digest is then shared on open networks to which other mail platforms also supply/consume (large mail platforms that have sufficient quantity of spam/ham themselves may run their own networks for competitive advantage) along with spam confidence scores (eg 75% likelihood of being spam) based on other tests, user feedback about false positives etc. It works extremely well because the Achilles heal of spam senders is that they have to send in bulk and so if the same digest is being reported across the world in a short length of time and other tests suggest it is spam then it gives a very high confidence level of it being exactly that.

if a user attempts to send the same body text, you assume
it's spam and filter it.

Spam filtering isn't that black and white. Scoring is used to give a likelihood of spam and only if a threshold is reached does it get filtered. Few, if any, spam rules are weighted enough to trip the filter on their own. Coupled with what's known as bayesian filtering there is usually a feedback loop which aims to improve the filtering over time even with the same set of rules. There is where the 'Not Spam' button in my large mail providers is particularly helpful - it doesn't just move it back into your Inbox from spam folder but also sends a small nudge to the filtering engines to suggest (not *absolutely say* - you can't trust users that much!) they might've got it wrong.

Fred wrote:
I have unfiltered mail delivered to me from my hosting service, I
simply reject anything not specifically to me (I do include stuff that
*might* be for me by allowing To: cgreen, To: cris, etc.) and then I
just filter it myself, by looking at it.

Much more convenient to have gmail do it completely
automatically and very reliably indeed.

Why?

Because all you have to do is use a gmail email address.

Hmm, after all the complaints I see about using gmail I'm not
convinced it's *that* "fit and forget"! :-)

By checking that nothing that should be
in the inbox is actually in the spam folder

What a hassle, having to look in the spam folder! :-)

--
Chris Green
·

"Chris Green" wrote in message
...
Fred wrote:
I have unfiltered mail delivered to me from my hosting service, I
simply reject anything not specifically to me (I do include stuff
that
*might* be for me by allowing To: cgreen, To: cris, etc.) and then I
just filter it myself, by looking at it.

Much more convenient to have gmail do it completely
automatically and very reliably indeed.

Why?

Because all you have to do is use a gmail email address.

Hmm, after all the complaints I see about using gmail I'm not
convinced it's *that* "fit and forget"! :-)

It has been for me with the spam filtering.

By checking that nothing that should be
in the inbox is actually in the spam folder

What a hassle, having to look in the spam folder! :-)

I dont bother any more now that I have checked its so reliable.

On Mon, 15 Feb 2021 04:06:55 +1100, Fred, better known as cantankerous
trolling senile geezer Rodent Speed, wrote:

FLUSH the trolling senile asshole's latest troll**** unread

--
Sqwertz to Rodent Speed:
"This is just a hunch, but I'm betting you're kinda an argumentative
asshole.
MID: