# $Id: praatje.txt 1247 2007-09-02 18:36:36Z joostvb $
# $URL: svn+ssh://agate.conuropsis.org/home/users/joostvb/srv/svn/www.mdcc.cx/trunk/mdcc.cx/email/praatje.txt $

Over deze tekst
===============

Gebaseerd op

 Date: Tue, 21 Aug 2007 20:33:19 +0200
 From: Joost van Baal
 To: bredaAlosc:nl
 Message-ID: <20070821183319.GH21107@nagy.mdcc.cx>
 Subject: [...] praatje over e-mail en greylisting en
        sender verification en zo op LOSC breda bijeenkomst van donderdag
        23 augustus [...])


Deze tekst bevat wat notities die de auteur gebruikte voor een presentatie voor
LOSC Breda ( http://losc.nl/Breda/ ) op donderdag 23 augustus 19:30-23:00 uur,
in Electron, Breda.


Inleiding
=========

Ik zal vooral aandacht besteden aan de manier waarop ik aan mij gerichtte spam
bestrijdt.  En dat is (ook) in mijn geval nodig, de cijfers voor 1 dag zijn:

 8.000 connects (maar op andere dagen zijn dat er 2.000)
   400 delivered
   100 niet-spam-na-spamassassin
    30 niet-spam-na-bogofilter

dus 90% (of 99%) is in mijn geval spam.

connects: 969 hosts, toppers:

     69 cpe-69-203-117-83.nyc.res.rr.com[69.203.117.83]
     75 unknown[206.162.202.213]
     77 host-85-27-28-206.brutele.be[85.27.28.206]
     84 etk211.neoplus.adsl.tpnet.pl[83.20.156.211]
     84 mailhost.terra.es[213.4.149.12]
     90 unknown[86.124.233.5]
     95 chello084010047073.chello.pl[84.10.47.73]
     96 mail.niitsmartserve.co.in[203.101.109.228]
     97 ALille-152-1-8-215.w82-127.abo.wanadoo.fr[82.127.180.215]
     99 p57A06F81.dip.t-dialin.net[87.160.111.129]
    288 herschel.uvt.nl[137.56.247.34]
    328 germain.uvt.nl[137.56.247.20]
    374 agnes.uvt.nl[137.56.247.33]
    777 74-138-151-219.dhcp.insightbb.com[74.138.151.219]
   1416 5aca630e.bb.sky.com[90.202.99.14]


De eerste SMTP-connectie: van het internet naar opal
====================================================

 $ host -t mx mdcc.cx
 mdcc.cx mail is handled by 4 opal.mdcc.cx.

Een SMTP-sessie ziet er bv uit als:

 joostvb@nagy:~% swaks --to joostvb-debianAmdcc:cx --from joostvbAdebian:org --server opal.mdcc.cx --helo nagy.mdcc.cx

=== Trying opal.mdcc.cx:25...
=== Connected to opal.mdcc.cx.
<-  220 opal.mdcc.cx ESMTP Postfix (Debian/GNU)
 -> EHLO nagy.mdcc.cx
<-  250-opal.mdcc.cx
<-  250-PIPELINING
<-  250-SIZE 10240000
<-  250-VRFY
<-  250-ETRN
<-  250-ENHANCEDSTATUSCODES
<-  250-8BITMIME
<-  250 DSN
 -> MAIL FROM:<joostvbAdebian:org>
<-  250 2.1.0 Ok
 -> RCPT TO:<joostvb-debianAmdcc:cx>
<-  250 2.1.5 Ok
 -> DATA
<-  354 End data with <CR><LF>.<CR><LF>
 -> Date: Sun, 05 Aug 2007 12:35:34 +0200
 -> To: joostvb-debianAmdcc:cx
 -> From: joostvbAdebian:org
 -> Subject: test
 ->
 -> This is a test mailing
 -> 
 -> .
<-  250 2.0.0 Ok: queued as 06022301A6
 -> QUIT
<-  221 2.0.0 Bye
=== Connection closed with remote host.

(telnet of nc werken ook:

 joostvb@agate:~% nc opal.mdcc.cx 25

)

Eerste veld antwoorden:

 2yz   Positive Completion reply
 3yz   Positive Intermediate reply
 4yz   Transient Negative Completion reply
 5yz   Permanent Negative Completion reply

Zie http://www.ietf.org/rfc/rfc2821.txt.

Het bestand opal.mdcc.cx:/etc/postfix/main.cf bevat:

 smtpd_client_restrictions = reject_unknown_client
 smtpd_helo_required = yes
 smtpd_helo_restrictions = reject_unknown_hostname
 smtpd_sender_restrictions = 
  permit_mynetworks,
  check_sender_access hash:/etc/postfix/smtpd_sender_restrictions,
  reject_unknown_sender_domain,
  check_client_access hash:/etc/postfix/smtpd_restrictions_access,
  reject_unverified_sender


smtpd_sender_restrictions:

 mdcc.cx OK

smtpd_restrictions_access:

 debian.org     OK
 uvt.nl         OK

(NB: mdcc.cx in smtpd_sender_restrictions is niet nodig als je slechts
van een beperkte set IPs mail From: @mdcc.cx zult verzenden.  In dat
geval kun je die smtpd_sender_restrictions beter weglaten. Spammers
kunnen dan niet meer ongestraft mail From: @mdcc.cx naar jou sturen.
(Bedankt LOSC Breda publiek voor deze tip!))

In de mail logs op opal vinden we:

Aug 23 06:31:56 opal postfix/smtpd[32258]: NOQUEUE: reject: RCPT from
 ppp85-141-230-97.pppoe.mtu-net.ru[85.141.230.97]: 450 4.7.1 <?g5>: Helo command
 rejected: Host not found; from=<jraabAucla.edu> to=<joostvb-debiannAmdcc:cx>
 proto=SMTP helo=<?g5>

Aug 23 06:28:09 opal postfix/smtpd[31986]: NOQUEUE: reject: RCPT from
 master.debian.org[70.103.162.29]: 450 4.1.8 <jramseyAlandstar.com>: Sender
 address rejected: Domain not found; from=<jramseyAlandstar.com>
 to=<joostvb-debianAmdcc:cx> proto=ESMTP helo=<master.debian.org>

Aug 23 06:28:28 opal postfix/smtpd[32249]: NOQUEUE: reject: RCPT from
 unknown[212.120.183.210]: 450 4.7.1 Client host rejected: cannot find your
 hostname, [212.120.183.210]; from=<jqsvdAhofstra.edu>
 to=<joostvb-www-topaz-7925Amdcc:cx> proto=SMTP helo=<co.durham.nc.us>

Aug 23 06:28:34 opal postfix/smtpd[32249]: NOQUEUE: reject: RCPT from
 64-60-51-132.static-ip.telepacific.net[64.60.51.132]: 450 4.1.7
 <ddhbqawngAbmps.hc.edu.tw>: Sender address rejected: undeliverable address:
 Host or domain name not found. Name service error for name=mail.bmps.hc.edu.tw
 type=A: Host not found; from=<ddhbqawngAbmps.hc.edu.tw>
 to=<joostvb-debian-bugs-20030221-8Amdcc:cx> proto=ESMTP helo=<mail3.g4si.com>

Aug 23 06:33:13 opal postfix/smtpd[32258]: NOQUEUE: reject: RCPT from
 ppp85-141-221-251.pppoe.mtu-net.ru[85.141.221.251]: 450 4.1.7
 <jr.jvalentineAtyrekamins.com>: Sender address rejected: undeliverable address:
 host tyrekamins.com.s6a1.psmtp.com[64.18.5.10] said: 550 5.1.1 User unknown (in
 reply to RCPT TO command); from=<jr.jvalentineAtyrekamins.com>
 to=<joostvb-nlo-hurd-20011221-223Amdcc:cx> proto=SMTP
 helo=<ppp85-141-221-251.pppoe.mtu-net.ru>


Maildrop en spamassassin
========================

opal.mdcc.cx, de machine die mail ontvangt voor het @mdcc.cx
emaildomein heeft:

 joostvb@opal:~% cat .forward
 | maildrop

De rest van de configuratie van die machine is te vinden op
http://mdcc.cx/arch/wd/ad1810-systems/opal/ ; alles wat onder
http://mdcc.cx/arch/wd/ad1810-systems/opal/etc/postfix/ staat, is de
systeemconfiguratie (dat staat dus in opal:/etc/postfix/.  De rest onder
http://mdcc.cx/arch/wd/ad1810-systems/opal/ leeft onder mijn
homedirectory daar.

Naast de bestanden die je daar kunt vinden is er nog joostvb@opal:.mailfilter;
dat ziet er ongeveer zo uit:

--------------

# maildrop filter file. see maildropfilter(5) and maildropex(7)

logfile $HOME/var/log/maildrop.log

if (/^X-XS4ALL-Spam: YES$/)
{
    to Mail/spam/inbox
}

# mail to @logreport.org, via uvt
if (/^X-Spam-Flag: YES$/ && \
    /^X-Virus-Scanned: by amavis.* at uvt.nl$/)
{
    to Mail/spam/inbox
}


#############################
# deal with spamtags added by unkown third parties
#############################

if (/^X-Spam-Flag: YES/)
{
    xfilter "reformail -a 'X-opal-X-Spam-Flag-from-unknown-party: YES'"
    xfilter "reformail -R X-Spam-Flag: X-old-Spam-Flag:"
}


#############################
# now we will calculate spamminess, and act upon it
#############################

# spamd listens on localhost:783
# Test it by running
#  joostvb@opal:~% date | spamc | less
# It is configured in joostvb@opal:~/.spamassassin/user_prefs

xfilter spamc

if (/^X-Spam-Flag: YES/)
{
    to Mail/spam/inbox
}

#############################
# deal with ham
#############################

# extension addresses in @mdcc.cx, defined in opal:/etc/aliases:
#  joostvb-postmaster, joostvb-abuse, joostvb-www, joostvb-webmaster
#  joostvb-clamav joostvb-amavis joostvb-root
# we _could_ handle these via ~/.forward-root e.a.


# SENDMAIL is /usr/sbin/sendmail -oi
# specifying
# to "!joostvb-opalAthuis:mdcc:cx"
# will give an empty env-from on forwarded mail
to "|$SENDMAIL -f $FROM joostvb-opalAthuis:mdcc:cx"


--------------------

en mijn crontab bevat daar:

 PATH=$HOME/bin:/usr/bin:/bin:/usr/sbin
 @monthly   mailrotate
 @daily     logrotate --state $HOME/var/lib/logrotate/status $HOME/etc/logrotate.conf



Op nagy.mdcc.cx, de machine die mail voor @thuis.mdcc.cx ontvangt, staat in
~joostvb/.mailfilter o.a.

---------------------

DEFAULT=Mail/inbox

xfilter 'bogofilter -p -u -e -v'

if (/^X-Bogosity: (Spam|Yes)/)
{
    to Mail/spam/inbox
}


if (/^X-Bogosity: Unsure/)
{
    to Mail/spam/inbox-unsure
}

---------------------

In joostvb@nagy:.muttrc staat o.a.:

---------------------

# esc-s
macro index \es "<pipe-entry>bogofilter -s -v\n<save-entry>=spam/missed\n" "bogofilter learn as spam, save in spam/missed"
# esc-h
macro index \eh "<pipe-entry>bogofilter -n -v\n<save-entry>=inbox\n" "bogofilter learn as ham, save in inbox"

---------------------



Complete mailheaders, na aflevering
===================================

Return-Path: master@capsaicin.example.com
X-Original-To: joostvb-opalAthuis:mdcc:cx
Delivered-To: joostvb-opalAthuis:mdcc:cx
Received: from stegun.mdcc.cx (stegun.mdcc.cx [IPv6:2001:888:1a9b:0:20a:dff:fed4:85d1])
        by nagy.mdcc.cx (Postfix) with ESMTP id 406EF2AC48D
        for <joostvb-opalAthuis:mdcc:cx>; Thu, 23 Aug 2007 08:26:44 +0200 (CEST)
Received: from opal.mdcc.cx (opal.mdcc.cx [80.69.71.45])
        by stegun.mdcc.cx (Postfix) with ESMTP id 22C937321
        for <joostvb-opalAthuis:mdcc:cx>; Thu, 23 Aug 2007 08:26:44 +0200 (CEST)
Received: from localhost (localhost.localdomain [127.0.0.1])
        by opal.mdcc.cx (Postfix) with ESMTP id 084DD30893
        for <joostvb-opalAthuis:mdcc:cx>; Thu, 23 Aug 2007 08:26:44 +0200 (CEST)
Received: from opal.mdcc.cx ([127.0.0.1])
        by localhost (opal.mdcc.cx [127.0.0.1]) (amavisd-new, port 10024)
        with ESMTP id W6LipKC0OHDp for <joostvb-opalAthuis:mdcc:cx>;
        Thu, 23 Aug 2007 08:26:43 +0200 (CEST)
Received: by opal.mdcc.cx (Postfix, from userid 1000)
        id EB49B30927; Thu, 23 Aug 2007 08:26:43 +0200 (CEST)
X-Spam-Checker-Version: SpamAssassin 3.1.7-deb (2006-10-05) on opal.mdcc.cx
X-Spam-Level:
X-Spam-Status: No, score=-2.0 required=5.0 tests=AWL,BAYES_00,
        FORGED_RCVD_HELO autolearn=ham version=3.1.7-deb
X-Original-To: joostvbAmdcc:cx
Delivered-To: joostvbAmdcc:cx
Received: from localhost (localhost.localdomain [127.0.0.1])
        by opal.mdcc.cx (Postfix) with ESMTP id 565C030923
        for <joostvbAmdcc:cx>; Thu, 23 Aug 2007 08:26:42 +0200 (CEST)
Received: from opal.mdcc.cx ([127.0.0.1])
        by localhost (opal.mdcc.cx [127.0.0.1]) (amavisd-new, port 10024)
        with ESMTP id 0ZpTl0lGilyh for <joostvbAmdcc:cx>;
        Thu, 23 Aug 2007 08:26:42 +0200 (CEST)
Received: from capsaicin.example.com (5.xs4all.nl [82.95.233.223])
        by opal.mdcc.cx (Postfix) with ESMTP id 2A9C530893
        for <joostvbAmdcc:cx>; Thu, 23 Aug 2007 08:26:42 +0200 (CEST)
Date: Thu, 23 Aug 2007 08:26:41 +0200
From: Lionel Elie Mamane <lionel@example.com>
To: Joost van Baal <joostvbAmdcc:cx>
Subject: Re: migration of joostvb's stuff from topaz to agate
Message-ID: <20070823062641.GA8831@capsaicin.example.com>
X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.1.5


Author, copyright
=================

Copyright (C) 2007 Joost van Baal <joostvb-email-praatje-losc-2353@mdcc.cx>

Redistribution and use, with or without modification, are permitted provided
that the above copyright notice, this condition and the following disclaimer
are retained.

This work is distributed in the hope that it will be useful, but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.