=pod =head1 NAME secure-list-patch - notes on Mailman patches for PGP support =head1 WARNING This document holds mainly internal stuff. Unless your name is Joost van Baal, it's contents is likely not interesting for you. One day, relevant stuff from this document will get included in the patch. =head1 INTRODUCTION No, there _really_ is NO publicly available (and find-able) guide on the internals of the Mailman code. Therefore, I am writing (a part of) one in this doc. =head1 REQUIREMENTS requirements (uit "Offerte Secure List Server") van Jan Meijer - Een lijstabonnee mag uitsluitend posten naar een mailinglist als de digitale handtekening geverifieerd kan worden als zijnde van een geauthoriseerde abonnee - Voor het sturen van versleutelde email versleutelt een lijstabonnee de email naar de public key van de betreffende lijst waarna de Secure List Service deze email decrypt en re-encrypt naar de public keys van de individuele abonnees. Het subscribermanagement (public-key administratie) in het geval van het SURFnet-PACT zal ingepast worden in het (geautomatiseerde) verwerken van SURFnet-PACT gegevens. - Na de uiteindelijke oplevering van alle onderdelen is het mogelijk om zowel RFC 2440 (OpenPGP) als RFC 2633 (S/MIME) berichten naar de SecureList te sturen waarbij de SecureList zorg draagt van omzetting naar het door de individuele subscribers gewenste formaat. More notes about what SURFnet wants: - Initially, it's fine to have all administrative work done by listadmins; members don't have to use the webinterface. Later, they probably _will_. - Op shamir worden alleen secure lijsten gehost, geen "vanilla" mailman lijsten. - Voorlopig zullen lijsten op shamir in securelist.surfnet.nl leven. Pas _veel_ later zullen er mogelijk ook externe lijsten komen. =head1 PATCHES patches staan op banach:/usr/local/src/mailman/ . Test ze op shamir. =head2 nah6 secure list patch joostvb@banach:/usr...rc/mailman/mailman-2.1.2% zcat ../secure-list-10-212.patch.gz | patch -p4 applies cleanly, apart from Defaults.py which should patch Defaults.py.in. ding lijkt niet te doen wat surfnet wil. Changed files: Mailman/Cgi/admin.py Mailman/Cgi/listinfo.py Mailman/Cgi/options.py Mailman/Commands/cmd_set.py Mailman/Defaults.py Mailman/Gui/__init__.py Mailman/Gui/SecureList.py Mailman/HTMLFormatter.py Mailman/MailList.py Mailman/PGPClass.py Mailman/PGPErrors.py Mailman/PGP.py Mailman/Queue/SecureRunner.py Mailman/SecureListHTMLFormat.py Mailman/SecureList.py scripts/post templates/en/listinfo.html templates/en/options.html Ding lijkt meer op een fork, ruikt vies. Mailman/PGP.py is van Frank J. Tobin, ftobin@neverending.org Lijkt dat deze een aparte queue wil hebben. Als ik in 1 keer kan decrypten en weer re-encrypten, dan heb ik geen aparte queue nodig: dan hoeft mail immers nooit in clear text te leven. t Ding maakt een queue genaamd 'secure', met een eigen SecureRunner. Denk dat ik ergens in OutgoingRunner decrypt en re-encrypt moet doen. Hm, of misschien aan GLOBAL_PIPELINE toevoegen: 'PGPCheck', (check wether the messages is encrypted; check signature (make a decrypted tmpfile for this)) 'SpamDetect', 'Approve', 'Replybot', 'Moderate', 'Hold', (raise exc -> does this move to other queue?) re-encrypt to moderator key? 'MimeDel', 'Emergency', 'Tagger', 'CalcRecips', 'AvoidDuplicates', 'Cleanse', (removes some headers) 'CookHeaders', (subject header cooking) 'PGPRecrypt', <- nieuw: decrypt, reencrypt to all recipients and sign 'ToDigest', (NOT supported for secure list) 'ToArchive', (NOT supported for secure list) 'ToUsenet', (NOT supported for secure list) 'AfterDelivery', 'Acknowledge', 'ToOutgoing', If a HoldMesssage error is raised: Queue/IncomingRunner.py: def _dopipeline(self, mlist, msg, msgdata, pipeline): while pipeline: try: sys.modules[modname].process(mlist, msg, msgdata) except Errors.HoldMessage: # Let the approval process take it from here. The message no # longer needs to be queued. return 0 return 0 =head2 Ben Laurie patch Patch #645297 _lijkt_ dichter in de buurt te komen. Die is tegen mailman-2.1, en applied redelijk clean. joostvb@banach:/usr.../src/mailman/mailman-2.1% cat ../mailman.patch.645297/mailman.patch | patch -p0 Changed files: configure.in Mailman/Cgi/options.py Mailman/Defaults.py.in Mailman/Gui/Privacy.py Mailman/Handlers/SMTPDirect.py Mailman/MailList.py Mailman/MemberAdaptor.py Mailman/Message.py Mailman/OldStyleMemberships.py Mailman/Queue/IncomingRunner.py Makefile.in templates/en/options.html Added files: PGP.py Defines: MailList.pgp_keys MailList.outgoing_pgp_policy MailList.incoming_pgp_required defines a per-member pgp policy Author: http://www.apache-ssl.org/ben.html Looks like a busy guy to me. =head2 Stefan Schlott patch http://medien.informatik.uni-ulm.de/~stefan/gpg-mailman.xhtml Ziet er _erg_ goed uit! copyright is raar: mailman-2.1.5/Mailman/Handlers/GpgDecrypt.py 2005-02-22 10:23:38.755106152 +0100 @@ -0,0 +1,183 @@ +# Copyright (C) 1998,1999,2000,2001,2002 by the Free Software Foundation, Inc. Applies absolutely clean: joostvb@banach:/usr...rc/mailman/mailman-2.1.5% zcat ../mailman-2.1.5-gpg_2005-02-22.diff.gz| patch -p1 patching file Mailman/Cgi/options.py patching file Mailman/Defaults.py.in patching file Mailman/GPGUtils.py patching file Mailman/Gui/Privacy.py patching file Mailman/Handlers/GpgDecrypt.py patching file Mailman/Handlers/GpgSMTPDirect.py patching file Mailman/MailList.py patching file Mailman/MemberAdaptor.py patching file Mailman/OldStyleMemberships.py patching file Mailman/mm_cfg.py.dist.in patching file Mailman/versions.py patching file README.GPG patching file bin/update patching file templates/en/options.html joostvb@banach:/usr/local/src/mailman% mv mailman-2.1.5 mailman-2.1.5-gpg_2005-02-22 shamir runs 2.1.5-7 En hoe werkt t met debian package? joostvb@banach:/usr...an/mailman/mailman-2.1.5% zcat ../../../mailman/mailman-2.1.5-gpg_2005-02-22.diff.gz | patch -p1 patching file Mailman/Cgi/options.py patching file Mailman/Defaults.py.in patching file Mailman/GPGUtils.py patching file Mailman/Gui/Privacy.py patching file Mailman/Handlers/GpgDecrypt.py patching file Mailman/Handlers/GpgSMTPDirect.py patching file Mailman/MailList.py patching file Mailman/MemberAdaptor.py patching file Mailman/OldStyleMemberships.py patching file Mailman/mm_cfg.py.dist.in patching file Mailman/versions.py patching file README.GPG patching file bin/update patching file templates/en/options.html joostvb@banach:/usr...an/mailman/mailman-2.1.5% vim debian/changelog joostvb@banach:/usr...an/mailman/mailman-2.1.5% dpkg-buildpackage -us -uc -rfakeroot http://non-gnu.uvt.nl/pub/mailman/ TODO Check mailman-developers draad over dit ding, interesante haken-en-ogen discussie. joostvb@shamir:/usr...local/src/debian/mailman% curl -O http://non-gnu.uvt.nl/pub/mailman/debian/sarge/mailman_2.1.5-7.1.gpg.1_i386.deb ImportError: No module named GnuPGInterface root@shamir:/usr...local/src/debian/mailman# apt-get install python-gnupginterface en nou doet mailman t ook weer. TODO: stick gpg patch in debian/patches/ . dit werkt: root@shamir:~# /usr/lib/mailman/bin/list_members Test-secure joostvb@uvt.nl dit niet: joostvb@shamir:~% w3m http://shamir.uvt.nl/cgi-bin/mailman/listinfo admin(18577): File "/var/lib/mailman/Mailman/GPGUtils.py", line 30, in ? admin(18577): import GnuPGInterface admin(18577): ImportError: No module named GnuPGInterface python-gnupginterface heeft /usr/lib/site-python/GnuPGInterface.py Default python include path vindt die. In our Defaults.py, we have: DEFAULT_GPG_POSTINGS_ALLOWED = No DEFAULT_GPG_MSG_DISTRIBUTION = No DEFAULT_GPG_MSG_SIGN = No pub 1024D/11F1F60B 2005-03-07 Test Really Secure List Key fingerprint = 1379 9540 3ED2 3C53 A954 8C76 4F6F E380 11F1 F60B sub 2048g/DDB70CD3 2005-03-07 joostvb@shamir:~% date | mail -s 'test, please ignore' test-really-secure@shamir.uvt.nl joostvb@shamir:~% cat < From: j.e.vanbaal@uvt.nl pipe heredoc> To: test-really-secure@shamir.uvt.nl pipe heredoc> Subject: test 2 pipe heredoc> pipe heredoc> blah pipe heredoc> EOT TODO: mail encrypted to list-key gets distributed encrypted to list-key. TODO: Mar 07 15:11:40 2005 (19333) Error importing keys: gpg: WARNING: unsafe permissions on homedir "/var/lib/mailman/lists/test-really-secure/gpg" See ``--no-permission-warning'' in gpg(1) root@shamir:/var...s/test-really-secure/gpg# gpg --homedir `pwd` --fingerprint gpg: WARNING: unsafe ownership on homedir "/var/lib/mailman/lists/test-really-secure/gpg" /var/lib/mailman/lists/test-really-secure/gpg/pubring.gpg --------------------------------------------------------- pub 1024D/11F1F60B 2005-03-07 Test Really Secure List Key fingerprint = 1379 9540 3ED2 3C53 A954 8C76 4F6F E380 11F1 F60B sub 2048g/DDB70CD3 2005-03-07 pub 1024D/88C6EDF6 2002-11-04 Joost van Baal Key fingerprint = 7177 F40B 051B 5793 8A0B E219 5F76 E17A 88C6 EDF6 uid Joost van Baal sub 1024g/450B4EE8 2002-11-04 [expires: 2035-05-01] seems replies are not encrypted against recipient: joostvb@banach:/usr...5-02-22/Mailman/Handlers% less GpgSMTPDirect.py for recip in msgdata['recips']: keyids=mlist.getGPGKeyIDs(recip) gh = GPGUtils.GPGHelper(mlist) ciphertext = gh.encryptSignMessage(plaintext,keyids) joostvb@banach:/usr...n-2.1.5/Mailman/Handlers% vim GpgSMTPDirect.py joostvb@banach:/usr...n-2.1.5/Mailman/Handlers% vim GpgDecrypt.py It _should_ log to /var/log/mailman/gpg , but apparently doesn't. Hrm.... Does it _reach_ our code block at all? Hrm, Defaults.py has non-gpg PIPELINE Status: it does do reencryption. TODO: it currently writes all plaintexts to log. Hrm... TODO: use joostvb@shamir as test user with own keypair TODO: check signature on posts: hack one of: Approve, Moderate (we have: 'SpamDetect', 'Approve', 'Replybot', 'GpgDecrypt', 'Moderate', 'Hold', ... ) Moved to SVN: /home/joostvb/svn/infix.uvt.nl/its-id/trunk/sources/mailman-ssls\ =head2 mailman releases mailman-2.1.tgz 31-Dec-2002 mailman-2.1.1.tgz 08-Feb-2003 mailman-2.1.2.tgz 27-Apr-2003 mailman-2.1.3.tgz 21-Nov-2003 mailman-2.1.4.tgz 31-Dec-2003 mailman-2.1.5.tgz 15-May-2004 mailman-2.1.6.tgz 31-May-2005 troll draait 2.1.5 security issues, dus up to date houden is niet overdreven. Misschien komt er nog een mailman-2.2. Er wordt al gewerkt aan mailman-3. (dd 2005-02) Beware to patch the right stuff: Subject: Re: [Mailman-Developers] new on list. From: Barry Warsaw To: Maat Date: Thu, 17 Feb 2005 14:41:07 -0500 Cc: mailman-developers@python.org The CVS tree on SF is the canonical tree, but remember that you want to work on the Release_2_1-maint branch, which is where MM2.1.x is released from. The trunk is what may someday be Mailman 2.2, but it is terribly bitrotted right now. Tokio has threatened to merge the branch back into the trunk, in preparation for a possible 2.2 release, but right now, you should consider the trunk unusable. =head1 PLANNING 0) richt shamir in kwart dag _bijna_ done 1) zoek uit wat de patch doet (en check of ie doet wat we willen) 2 dagen kijk welke files gepatch-ed worden, en zoek uit wat die files doen 1a) mail 2 patch hakkers, cc mm devel en knakker op http://lists.indymedia.org/pipermail/listwork/2004-August/0821-q9.html . en dan, _ALS_ ie doet wat we willen .) zorg dat ie tegen laatste mailman werkt 1 dag .) bak er een debian package van, check of t niet conflicteert met debian patches 1 dag mailman 2.1.5-7 heeft _*38*_ patches in debian/patches (gebruikt dpatch) .) probeer patch in debian (of upstream) geintegreerd te krijgen halve dag 5 dagen (als ALLES goed gaat) 1 april 2005 PGP klaar daarna s/mime =head1 MAILMAN CODE Er is een STYLEGUIDE.txt is Mailman CVS. Patches moeten naar SF gestuurd worden, en, voor oude Mailman's, in wiki bedocumenteerd worden. MailList.py - a Mailman mailinglist object Message.py - Mailman Message object, subclass of standard email.Message Post.py - inject a message in the Mailman queue Queue/ - Mailman's own queue MemberAdaptor.py - defines what a mailing list member is, empty framework Cgi/options.py - handle members tweaking subscribtion options via web interface interfaces: =head2 /var/lib/mailman/mailman /var/lib/mailman/mail/mailman {post,confirm,join,leave,request,subscribe,unsubscribe} built from mail-wrapper.c invokes /var/lib/mailman/scripts/{post,confirm,join,leave,request,subscribe,unsubscribe} these invoke Mailman.Queue.sbcache.get_switchboard.enqueue , with the appropriate queue taken from mm_cfg. code is in Mailman/Queue/Switchboard.py : _Switchboard.enqueue() It seems all messages are enqueued, no matter stuff as post-policy and anti-spam measures. Same for dequeue. =head2 qrunner qrunner is started on boot with "/usr/lib/mailman/bin/mailmanctl -s start" See "mailmanctl --help" for usage info. Typically, one sees: python mailmanctl -s start \_ python qrunner --runner=ArchRunner:0:1 -s \_ python qrunner --runner=BounceRunner:0:1 -s \_ python qrunner --runner=CommandRunner:0:1 -s \_ python qrunner --runner=IncomingRunner:0:1 -s \_ python qrunner --runner=NewsRunner:0:1 -s \_ python qrunner --runner=OutgoingRunner:0:1 -s \_ python qrunner --runner=VirginRunner:0:1 -s \_ python qrunner --runner=RetryRunner:0:1 -s See "qrunner --help" for qrunner usage. We have these queues: root@brouwer:/var/lib/mailman/qfiles# ls archive/ bounces/ commands/ in/ news/ out/ retry/ shunt/ virgin/ Defaults.py has: ('ArchRunner', 1), # messages for the archiver ('BounceRunner', 1), # for processing the qfile/bounces directory ('CommandRunner', 1), # commands and bounces from the outside world ('IncomingRunner', 1), # posts from the outside world ('NewsRunner', 1), # outgoing messages to the nntpd ('OutgoingRunner', 1), # outgoing messages to the smtpd ('VirginRunner', 1), # internally crafted (virgin birth) messages ('RetryRunner', 1), # retry temporarily failed deliveries Defaults.py defines the queues. Each qrunner process invokes Mailman.Queue..qrclass.run(). Queue/Runner defines run(): def run(self): filecnt = self._oneloop() def _oneloop(self): files = self._switchboard.files() for filebase in files: msg, msgdata = self._switchboard.dequeue(filebase) self._onefile(msg, msgdata) def _onefile(self, msg, msgdata): listname = msgdata.get('listname') mlist = self._open_list(listname) keepqueued = self._dispose(mlist, msg, msgdata) if keepqueued: self._switchboard.enqueue(msg, msgdata) def _dispose(self, mlist, msg, msgdata): """Subclasses must provide implementation for this method.""" raise NotImplementedError It seems def _dispose(self, mlist, msg, msgdata): pipeline = self._get_pipeline(mlist, msg, msgdata) more = self._dopipeline(mlist, msg, msgdata, pipeline) def _get_pipeline(self, mlist, msg, msgdata): return msgdata.get('pipeline', getattr(mlist, 'pipeline', mm_cfg.GLOBAL_PIPELINE))[:] def _dopipeline(self, mlist, msg, msgdata, pipeline): handler = pipeline.pop(0) modname = 'Mailman.Handlers.' + handler sys.modules[modname].process(mlist, msg, msgdata) is the gist of Queue/IncomingRunner.py Defaults.py: # All `normal' messages which are delivered to the entire list membership go # through this pipeline of handler modules. Lists themselves can override the # global pipeline by defining a `pipeline' attribute. GLOBAL_PIPELINE = [ # These are the modules that do tasks common to all delivery paths. 'SpamDetect', 'Approve', 'Replybot', 'Moderate', 'Hold', 'MimeDel', 'Emergency', 'Tagger', 'CalcRecips', 'AvoidDuplicates', 'Cleanse', 'CookHeaders', # And now we send the message to the digest mbox file, and to the arch and # news queues. Runners will provide further processing of the message, # specific to those delivery paths. 'ToDigest', 'ToArchive', 'ToUsenet', # Now we'll do a few extra things specific to the member delivery # (outgoing) path, finally leaving the message in the outgoing queue. 'AfterDelivery', 'Acknowledge', 'ToOutgoing', ] GLOBAL_PIPELINE is used by IncomingRunner only. The ToOutgoing module is only for use by the IncomingRunner for delivering messages posted to the list membership. Anything else that needs to go out to some recipient should just be placed in the out queue directly. Handling moderation and spamdetection is done in Handlers/{Moderate,SpamDetect}.py OutgoingRunner handles failed deliveries to some recipients. Passing outgoing mail to sendmail is done in Handlers.SMTPDirect.process() =head1 GENERAL PYTHON STUFF Extracting docstrings from a .py file is quite often impossible if the .py file is not proberly installed on the system: joostvb@banach:~/cv.../mailman/mailman/Mailman% pwd /home/joostvb/cvs/sf/mailman/mailman/Mailman % python Python 2.3.4 (#2, Jan 5 2005, 08:24:51) [GCC 3.3.5 (Debian 1:3.3.5-5)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import MailList Traceback (most recent call last): File "", line 1, in ? File "./MailList.py", line 42, in ? from Mailman import mm_cfg ImportError: No module named Mailman >>> print MailList.__doc__ The class representing a Mailman mailing list. Mixes in many task-specific classes. >>> help(MailList) >>> help() import-ing MailList second time gives success. Duh. >>> print MailList.ApprovedAddMember.__doc__ Traceback (most recent call last): File "", line 1, in ? AttributeError: 'module' object has no attribute 'ApprovedAddMember' >>> print MailList.__doc__ The class representing a Mailman mailing list. Mixes in many task-specific classes. >>> ???? =head1 PYTHON AND GNUPG http://py-gnupg.sourceforge.net/ ; python-gnupginterface - Python interface to GnuPG (GPG) by Frank J. Tobin GnuPGInterface-0.3.2.tar.gz 2002-01-10 is latest release. It contains GnuPGInterface.py , see banach:/usr/local/src/gnupginterface/ =head1 INSTALLING AND CONFIGURING MAILMAN Brouwer runs mailman with apache2. Let's run apache2 on shamir too. The following packages will be REMOVED: apache* apache-common* The following NEW packages will be installed: apache2-common apache2-mpm-worker libapr0 libgcrypt11 libgnutls11 libgpg-error0 libldap2 liblzo1 libopencdk8 libpcre3 libsasl2 libtasn1-2 openssl ssl-cert The following packages will be upgraded: bzip2 console-common console-data console-tools curl diff dpkg dselect gcc-3.3-base grub iproute libbz2-1.0 libconsole libcurl3 libdb3 libdb4.2 libsensors3 libstdc++5 login mailman memtest86 netbase passwd perl perl-base perl-modules postfix python2.3 strace ucf zsh See also lovelace and brouwer for apache2 configs. # as user list, run /usr/lib/mailman/bin/genaliases to update virtual-mailman.db root@shamir:/var/lib/mailman# ./bin/genaliases (generates mailman list and test-secure list: nice) root@shamir:/var/lib/mailman/data# chown list:daemon aliases* Can subscribe to test-secure list. =head1 DEBUGGING LIVE MAILMAN =head2 feeding a message from a file Using the standard python email package: >>> import email >>> fp = file("tmp/mailtje.mail", 'r') >>> msg = email.message_from_file(fp) >>> dir(msg) >>> print msg.get_charsets() ['us-ascii'] >>> print msg.get_payload() ja echt, dit is een mailtje In Queue/Switchboard.py dequeue() the email object is created (when messages are not stored as pickles) from Mailman import Message msg = email.message_from_string(msg, Message.Message) Using the Mailman.Message stuff: joostvb@shamir:~% cat PYTHONSTARTUP import paths from Mailman import mm_cfg import sys sys.path.append('/usr/lib/mailman/Mailman/Handlers') sys.path.append('/usr/lib/mailman/Mailman/') import Utils joostvb@shamir:/usr/lib/mailman/bin% PYTHONSTARTUP=~/PYTHONSTARTUP python >>> from Mailman import Message >>> fp = file("/home/joostvb/tmp/mailtje.mail") >>> msg = fp.read() >>> import email >>> msg = email.message_from_string(msg, Message.Message) Mailman.Message defines extra methods __version__, get_sender() and get_senders(), not present in email.email: >>> print msg.get_senders() ['j.e.vanbaal@uvt.nl'] =head2 ripping a message from the queue A log of what we've done to debug a live mailman system. joostvb@shamir:~% date | mail -s 'test 2' test-secure@shamir.uvt.nl ends up in shamir:/var/lib/mailman/qfiles/in/1109598085.603312+fa2eab606f5dcbb54ffee413789c4b5a12550001.pck when no qrunner. >>> import pickle >>> reader = pickle.load(open('save.p')) >>> reader.readline() joostvb@shamir:~% mv 1109598085.603312+fa2eab606f5dcbb54ffee413789c4b5a12550001.pck mailtje.pck joostvb@shamir:~% python Python 2.3.5 (#2, Feb 9 2005, 00:38:15) [GCC 3.3.5 (Debian 1:3.3.5-8)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import pickle >>> m = pickle.load(open('mailtje.pck')) >>> print m From joostvb@uvt.nl Mon Feb 28 14:41:25 2005 Return-Path: X-Original-To: test-secure@shamir.uvt.nl Delivered-To: test-secure@shamir.uvt.nl Received: by shamir.uvt.nl (Postfix, from userid 1000) id 769CA148; Mon, 28 Feb 2005 14:41:25 +0100 (CET) To: test-secure@uvt.nl Subject: test 2 Message-Id: <20050228134125.769CA148@shamir.uvt.nl> Date: Mon, 28 Feb 2005 14:41:25 +0100 (CET) From: joostvb@uvt.nl (Joost van Baal) Mon Feb 28 14:41:25 CET 2005 >>> type(m) >>> isinstance(m,str) True root@shamir:/var...ailman/lists/test-secure# cp pending.pck ~joostvb/ >>> p = pickle.load(open('pending.pck')) >>> type(p) >>> print p {'evictions': {}, 'version': 2} root@shamir:~# /usr/lib/mailman/bin/qrunner --runner=IncomingRunner:0:1 -s ^C ./out/1109598085.603312+795be568131e3f5287cf87d45afa8dcb681e41fa.pck root@shamir:/var/lib/mailman/qfiles# cp out/1109598085.603312+795be568131e3f5287cf87d45afa8dcb681e41fa.pck ~joostvb/mailtje.pck joostvb@shamir:/usr/lib/mailman/bin% python joostvb@shamir:/usr/lib/mailman/bin% python Python 2.3.5 (#2, Feb 9 2005, 00:38:15) [GCC 3.3.5 (Debian 1:3.3.5-8)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import paths >>> from Mailman import mm_cfg >>> import pickle >>> m = pickle.load(open('/home/joostvb/mailtje.pck')) >>> type(m) >>> dir(m) ['__contains__', '__delitem__', '__doc__', '__getitem__', '__init__', '__len__', '__module__', '__repr__', '__setitem__', '__setstate__', '__str__', '__version__', '_charset', '_default_type', '_get_params_preserve', '_headers', '_payload', '_unixfrom', 'add_header', 'add_payload', 'as_string', 'attach', 'del_param', 'epilogue', 'get', 'get_all', 'get_boundary', 'get_charset', 'get_charsets', 'get_content_charset', 'get_content_maintype', 'get_content_subtype', 'get_content_type', 'get_default_type', 'get_filename', 'get_main_type', 'get_param', 'get_params', 'get_payload', 'get_sender', 'get_senders', 'get_subtype', 'get_type', 'get_unixfrom', 'has_key', 'is_multipart', 'items', 'keys', 'preamble', 'replace_header', 'set_boundary', 'set_charset', 'set_default_type', 'set_param', 'set_payload', 'set_type', 'set_unixfrom', 'values', 'walk'] >>> print m._headers [('Return-Path', ''), ('X-Original-To', 'test-secure@shamir.uvt.nl'), ('Delivered-To', 'test-secure@shamir.uvt.nl'), ('Received', 'by shamir.uvt.nl (Postfix, from userid 1000)\n\tid 769CA148; Mon, 28 Feb 2005 14:41:25 +0100 (CET)'), ('To', 'test-secure@uvt.nl'), ('Message-Id', '<20050228134125.769CA148@shamir.uvt.nl>'), ('Date', 'Mon, 28 Feb 2005 14:41:25 +0100 (CET)'), ('From', 'joostvb@uvt.nl (Joost van Baal)'), ('Subject', ), ('X-BeenThere', 'test-secure@shamir.uvt.nl'), ('X-Mailman-Version', '2.1.5'), ('Precedence', 'list'), ('List-Id', 'test-secure.shamir.uvt.nl'), ('List-Unsubscribe', ',\n\t'), ('List-Archive', ''), ('List-Post', ''), ('List-Help', ''), ('List-Subscribe', ',\n\t')] root@shamir:~# /usr/lib/mailman/bin/qrunner --runner=OutgoingRunner:0:1 -s Now it's in archive/1109598085.603312+9... Starting master qrunner purges it from there too. generic_nonmember_action : hold joostvb@shamir:~% cat < From: j.e.vanbaal@uvt.nl pipe heredoc> To: test-secure@shamir.uvt.nl pipe heredoc> Subject: test3 pipe heredoc> pipe heredoc> blah pipe heredoc> EOT root@shamir:/var/lib/mailman/qfiles# find . . ./virgin ./archive ./shunt ./bounces ./commands ./in ./news ./retry ./out Gets stored in root@shamir:/var/lib/mailman/data# ls -l heldmsg-test-secure-1.pck -rw-rw-r-- 1 list list 665 Feb 28 15:28 heldmsg-test-secure-1.pck . Approval request, as sent to list admin, contains the message. Hrm. Re-encrypt just right before sending out? Can we verify signature without decrypting? No we can't! However, we have to verify before we can decide what to do with the message: bounce/defer or distribute. Ergo: re-encrypt a.s.a.p! Otoh: is it bad for the message to be clear-text while in the queue? Yes! If it's deferred, it's not interesting anyway? Yes, it _is_ interesting! What if member erroneously signs with wrong key, and encrypts? Such a message should be kept protected! So: 1) decrypt to check, keep encrypted; 2) wrong sig? then: save a copy of original message, decrypt, sign with listkey, encrypt to listadmin key, sent for acknowledgement and sent original message to poster 2a) if not acknowledged, then: warn sender: mail back original message? exit else: go on 3) decrypt, sign with listkey, encrypt to all members (hide recipient key, in order not to expose members to recipient?) Another handy trick: root@shamir:~# /usr/sbin/config_list -o - test-secure | less =head2 more python debugging joostvb@shamir:~/sv...ls/mailman-2.1.5/Mailman% PYTHONPATH=/home/joostvb/svn/infix.uvt.nl/its-id/trunk/sources/mailman-ssls/mailman-2.1.5/Mailman/Handlers:/usr/lib/mailman/bin; export PYTHONPATH joostvb@shamir:~/sv...ls/mailman-2.1.5/Mailman% python Python 2.3.5 (#2, Feb 9 2005, 00:38:15) [GCC 3.3.5 (Debian 1:3.3.5-8)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import paths >>> from Mailman import mm_cfg >>> import Moderate Traceback (most recent call last): File "", line 1, in ? File "/home/joostvb/svn/infix.uvt.nl/its-id/trunk/sources/mailman-ssls/mailman-2.1.5/Mailman/Handlers/Moderate.py", line 85 else ^ SyntaxError: invalid syntax >>> joostvb@shamir:/usr/lib/mailman/bin% python Python 2.3.5 (#2, Feb 9 2005, 00:38:15) [GCC 3.3.5 (Debian 1:3.3.5-8)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import paths >>> from Mailman import mm_cfg >>> import pickle >>> m = pickle.load(open('/home/joostvb/mailtje.pck')) >>> >>> import sys >>> sys.path.append('/usr/lib/mailman/Mailman/Handlers') >>> import Moderate >>> sys.path.append('/usr/lib/mailman/Mailman/') >>> import Utils >>> Utils.list_names() >>> from MailList import MailList >>> mlist = MailList('test-secure', lock=False) IOError: [Errno 13] Permission denied: '/var/lib/mailman/lists/test-secure/config.pck' # adduser joostvb list >>> import GPGUtils >>> gh = GPGUtils.GPGHelper(mlist) key_ids = gh.verifyMessage(msg) >>> m = pickle.load(open('/home/joostvb/mailtje.pck')) >>> key_ids = gh.verifyMessage(m) (we have a gpg process) ^D >>> key_ids = gh.verifyMessage(m) >>> key_ids >>> type(key_ids) >>> print key_ids None joostvb@shamir:/usr/lib/mailman/bin% PYTHONSTARTUP=~/PYTHONSTARTUP python >>> key_ids = gh.verifyMessage(m) >>> key_ids [] >>> type(key_ids) >>> mooi zo. Beware! One might have to restart mailman daemon after push of meuk. root@shamir:/usr...mailman/Mailman/Handlers# cp ~joostvb/svn/infix.uvt.nl/its-id/trunk/sources/mailman-ssls/mailman-2.1.5/Mailman/GPGUtils.py /usr/lib/mailman/Mailman/ gpg --homedir /var/lib/mailman/lists/test-secure/gpg --edit-key 36720983 Secret key is available. gpg: checking the trustdb gpg: no ultimately trusted keys found joostvb@shamir:/usr/lib/mailman/bin% PYTHONSTARTUP=~/PYTHONSTARTUP python >>> dir(mlist) >>> mlist.web_page_url 'http://shamir.uvt.nl/cgi-bin/mailman/' =head2 viewing a list joostvb@shamir:/usr/lib/mailman/bin% python Python 2.3.5 (#2, Sep 4 2005, 22:01:42) [GCC 3.3.5 (Debian 1:3.3.5-13)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import paths >>> from Mailman import mm_cfg >>> import pickle >>> import sys >>> sys.path.append('/usr/lib/mailman/Mailman/') >>> import Utils >>> from MailList import MailList >>> mlist = MailList('test-secure', lock=False) >>> dir(mlist) >>> members = mlist.members.keys() >>> mlist.gpgkeyids {'guus+securelist@s': ['0x5d8cda7b'], 'joostvb@xxx.xxx': ['0x0c14ff8c'], 'j.e.vanbaal@l': ['0x0b86b067']} >>> keys = mlist.gpgkeys >>> type(keys) >>> keys.keys() ['guus+securelist@s', 'joostvb@nl' ... =head1 BUILDING THE SSLS PATCH (AND MORE) FROM OUR DARCS REPO joostvb@banach:~/da...ailman-ssls/mailman-ssls% darcs pull joostvb@banach:~/da...ailman-ssls/mailman-ssls% make patch Now /usr/local/src/mailman contains mailman-2.1.6-ssls_2005-10-28.patch and a directory mailman-2.1.6-ssls_2005-10-28 with patch applied. Run joostvb@banach:/usr...an-2.1.6-ssls_2005-10-28% ./configure --prefix=/var/lib/mailman --with-username=list --with-groupname=list --with-mail-gid=daemon --with-cgi-gid=www-data --without-permcheck --with-mailhost=localhost --with-urlhost=localhost to generate Mailman/Defaults.py, Mailman/mm_cfg.py.dist (and other stuff). =head1 TEST LOG =head2 The test lists The test-secure list has settings: gpg_postings_allowed Yes (encrypt post to listkey) gpg_msg_distribution Force (distribute encypted) gpg_post_sign Force (should posts be signed) gpg_msg_sign Yes (distribute signed) The test-medium list has settings: gpg_postings_allowed Yes gpg_msg_distribution Yes gpg_post_sign Force The test-vanilla list has settings: gpg_postings_allowed No gpg_msg_distribution No gpg_post_sign No gpg_msg_sign No FIXME : gpg_msg_distribution does not do what SURFnet wants. SURFnet wants: if message posted unencrypted, distribute unencrypted. =head2 gpg_post_sign var is Yes: unsigned: ok RECHECK (hold) signed: ok RECHECK (distributed) var is No: ok var is Force: unsigned: ok (auto-discard) signed: ok (distributed (encrypted)) =head2 gpg_postings_allowed var is Yes: signed & encrypted: ok =head2 fixing subscribing via webinterface At 2005-04-12 (and probably way before), subscribing via webinterface was broken. root@shamir:/usr.../lib/mailman/Mailman/Cgi# mv subscribe.py subscribe.py.bak fixed mm_cfg.py -> s/http/https/ . root@shamir:/var/lib/mailman/bin# ./withlist -l -r fix_url test-secure -v =head2 2005-07-22 Upgrading from mailman 2.1.5-7.1.gpg.11 to mailman-ssls_2.1.6-0.1. root@shamir:/var/lib# tar zcf mailman.tar.gz mailman root@shamir:/var/lib# mv mailman.tar.gz ~/ root@shamir:~# dpkg --remove mailman (Reading database ... 18419 files and directories currently installed.) Removing mailman ... Shutting down Mailman's master qrunner hrm, dat fl*kkert gelukkig niet mn lists weg ( /var/lib/mailman/lists ) root@shamir:~# dpkg -i /usr/local/src/debian/mailman/mailman-ssls_2.1.6-0.1_i386.deb Selecting previously deselected package mailman-ssls. (Reading database ... 15390 files and directories currently installed.) Unpacking mailman-ssls (from .../mailman-ssls_2.1.6-0.1_i386.deb) ... Setting up mailman-ssls (2.1.6-0.1) ... Starting Mailman's master qrunner. testlog: plain to test-vanilla: ok plain to test-medium: ok (discard) signed to test-medium: ok (crypt) crypted to test-medium: ok (crypt) plain to test-secure: ok (discard) signed to test-secure: ok (discard) crypt to test-secure: ok (crypt) =head2 2005-10-28 Upgrade to mailman-2.1.6-ssls_2005-10-28.patch. Seems not obviously broken. =head2 2006-01-12 Upgrade from 2.1.6-0.1 to 2.1.6-0.2. See log.pod on private repo. signed/crypted with 'joostvb@shamir.uvt.nl': ['0x0c14ff8c'] to test-secure: ok! (crypt) =head1 adding new list root@shamir:~# newlist --quiet ssls-devel j.e.vanbaal@uvt.nl root@shamir:/var/lib/mailman/lists# mkdir ssls-devel/gpg root@shamir:/var/lib/mailman/lists# chown www-data.list ssls-devel/gpg root@shamir:/var/lib/mailman/lists# chmod o-rx ssls-devel/gpg root@shamir:/var/lib/mailman/lists# chmod g+w ssls-devel/gpg root@shamir:~# gpg --homedir /var/lib/mailman/lists/ssls-devel/gpg --gen-key joostvb@shamir:~% gpg --import - joostvb@shamir:~% gpg --send-key 9450991A =head1 VERSION $Id: secure-list-patch.pod 11392 2005-07-22 13:31:53Z joostvb $ $URL: https://pong.uvt.nl/its-unix/systems/shamir/doc/secure-list-patch.pod $ =head1 COPYRIGHT Copyright (C) 2005 Tilburg University =head1 AUTHOR Joost van Baal =cut