“Considered Harmful”, Spam, and SPF

So lately we’re getting tons of spam. Any sense of the word ‘we’ you can come up with, we are getting it. The stuff that seems to keep making it through everything tends to be image spam (can’t do bayesian stuff to it, no text) for stock scams (no need to put a URL in the content of the email, which we would catch and block).

So at first I was considering running OCR on all email that came in and had images on it – but that’s really scary. It would mean having the computer figure out that there’s text in every image and scanning it out and then running SpamAssassin or whatever on that image. There seems to be one plugin for this and it seems crappy – it has to filter your image through an image converter, then into an ocr package, then the text that comes out gets checked against a static list. Lame. I would prefer the text be fed into SpamAssassin or something, so we get a little more flexibility out of the setup. But even then – you just start making swirlyer text, more obfuscated, and your OCR plug-in won’t be able to read it.

But I decided to look into some other options – and one I decided to implement is called SPF. Sender Policy Framework, it’s been extended by Microsoft into some sender-ID proposal. You check DNS to see if someone who’s sending you mail is listed in a TXT record to be ‘authorized’ to send mail for that domain. If they aren’t, you can bounce it.

Now, ultimately, the spam problem is a legal problem, that is impossible to enforce because of all the forging that goes on. Pump-and-dump stock schemes are an FTC issue, for example. But we can’t tell who’s spamming us because they’re sending through zombie networks with forged ‘from’ addresses. If we knew who they were, we could refer the FTC to them, and they could attack them from that direction. SPF _may_ end up helping with that kind of thing. Maybe.

But today I had to wade through a ton of articles begging me not to implement SPF because of the horror and tragedy that would ensue. Oh no! But, as before, “X Considered Harmful” is just another way to cause a knee-jerk reaction. If some domain out there in the world chooses to publish SPF records for their domain, and you choose to obey those SPF records, it’s not a big deal. If you don’t like SPF records, don’t publish any, or publish a “+all” record if you want to be a dick about it. Why go on a tirade? If some guy publishes a record and fucks up his email, isn’t that his problem, not yours?

Now, that being said, there are problems with this SPF thing, among which are handling for forwarders. But the bulk of the technical disagreements here don’t seem valid. In the modern era, there are no open relays anymore. If you relay mail, you relay it for someone. Whoever ‘someone’ is, if they want, they can publish an SPF record that says so. If you’re trying to do some tricky thing with moving around and sending mail from dynamic addresses, you’re likely getting marked as spam anyway because of your address dynamicness.

But forwarders seems to be a legit problem. Domain A sends mail to Domain B. foo@b.com forwards to bar@c.com. So now we have the mail server at b.com sending mail from somebody at a.com to c.com. Wait, that’s not a problem, is it? No, it is – imagine c.com checks the SPF record – mail is coming from Domain A, so it will be checking A’s SPF record. A’s SPF record says that A will only send mail from A’s server. So that’s the infamous Forward problem. Eh, not good. But still, it’s A’s problem, not my problem (being Mr. C). Shit. Basically, the actions of the recipient on server B will affect whether or not his email will forward properly. He goes into his account settings, says ‘forward to server C’, and mysteriously finds that some messages (from servers other than A, who don’t use SPF) get through, whereas others (from servers like A, who _do_ use SPF with some kind of restrictive setting), will get mysteriously bounced or marked as spam. Well…I dunno. The user at C who changed his forward on server B is going to find his mail kinda does get delivered, kinda doesn’t. Depends on who it comes from. And that’s because I (owner of server C) turned on SPF checks. It is only in the case of a ‘forward’, and it can be fixed by mangling the envelope sender so it appears to be from the B server’s domain…but…ugh. In any case, it’s a setting on A’s server that seems to cause the problem. If the user on C isn’t getting mail from A that’s going through his forward at B, well, don’t do the forward, or use a new-style forwarder thingee.

Shit, maybe I do have to do some kind of OCR thing after all. Ugh. I hate this crap. And after I _manually_ went and applied patches onto qmail. I need a new mailserver, too.

3 thoughts on ““Considered Harmful”, Spam, and SPF”

  1. We use NYI.net for email services. I was wondering if bringing my mail server back to our location and handling our mail on my own would cut down on the spam…but you seem to shoot that down without mucho work and twitchy software (you know we deal with TONS of images).
    The best we have going now is to set rules that bounce mail not included in the list of known clients and that seems to trick spam servers into believing they arent hitting a legit server, only problem is we get much email from unknowns who get bounced too.
    I was currently looking for a better way to do it per machine and email client (since the servers dont reside with us). But this is all very very bad…sigh

  2. Brady,
    Spam 2.0 and the rise have given me and rick huge headaches!
    The kneejerk reactionsby some of our new clients have caused our mail to be blocked!
    (the people i work for refuse to admit that they need to get me in touch with the other tech dept and they keep insisting its my ricks or nyi’s fault but thats another story)
    The world of email is about to die I think, i.m. is the future but not like this possibly txting too.

  3. So the email thing is blowing up way out of proportion.
    Once again I find myself trying to fix a problem that is completly not mine. We are currently setting up a SPF record to see if we can circumvent the email issue with the client (once again I find myself not able to convince someone to let me talk to the tech dept of our clients, sigh).
    The fact that its 2 major companies makes the sales people scared I think.
    Maybe they dont have I.T. and thats why my company makes me help these damn people all the time (for free!!!)
    I will keep you posted…

Leave a Reply

Your email address will not be published. Required fields are marked *