Migrating from Evolution to Gnus + notmuch (part 3/x)

After having migrated my mail from Evolution’s storage to a local dovecot imap server, I’ve now started the migration to Gnus and notmuch.

I’ll then now manage my mail with Emacs 🙂

The migration to notmuch allows me to index all the threads even though they happen to cross the boundaries of the folders I’ve setup with dovecot’s sieve (see previous post). Still, in Gnus, I can still read the contents of the mail folders, for instance each mailing-list at a time. There are quite a few configuration difficulties so that Gnus and notmuch-emacs can work together well, which I’ll try and document if I find enough time.

I’ve also added a notification script that will notify my Gnome desktop of new mail (which will be the subject of a dedicated post).

While at it, I’ve also installed msmtp, which allows me to use different SMTP servers, depending on the different mail aliases I’ve used (in a symetrical way to what fetchmail does to fetch all mail from my different mail accounts).

I’m now able to integrate my mails with my org-mode setup, referencing Gnus mails in org notes, or tagging in notmuch with same tag names as in org-mode.

I’m now fully Emacs operated, and hope to gain in productivity 🙂

Stay tuned for more details

Migrating mail from Evolution local storage to Evolution + local Imap (dovecot with Maildir) 2/X

See previous post for the rationale.

I’ve then started migrating my mail using evolution.

First I’ve installed dovecot and set it up so as to store my mail inside my $HOME/Maildir/ dir. Evolution will run dovecot on demand through a pipe, instead of through network access like with regular remote IMAP servers. So dovecot won’t be started as a daemon. I’ve been inspired by Roland’s setup for this (even though I’not using offlineimap).

Inside Evolution, I’ve configured a second mail profile, for this local IMAP server, using the custom command to connect to server :
"MAIL=maildir:$HOME/Maildir /usr/lib/dovecot/imap"

I’ve migrated my existing mail folders using “copy to” function on all top-level mail folders from the local evolution storage, to copy them to the IMAP folders. Note that some folders that include a dot (‘.’) need to be renamed, as the dot is a path separator for IMAP.

I’ve adapted Evolution’s vfolders definitions so as to take into account the “active remote folders” instead of the local ones, and that’s it, the mail is migrated.

I’m using fetchmail + procmail to fetch mail from remote servers and to deliver it to the Maildirs, using dovecot’s deliver program, with something like :

DELIVER="/usr/lib/dovecot/deliver"

:0: w
| $DELIVER

I’ve migrated the filters of evolution to “server-side” Sieve rules so that dovecot’s deliver sorts incoming mail in the right Maildirs. The dovecot config needs to be adapted to activate the sieve plugin (dovecot-sieve Debian package), in /etc/dovecot/conf.d/15-lda.conf :

protocol lda {
# Space separated list of plugins to load (default is global mail_plugins).
mail_plugins = $mail_plugins sieve
log_path = /tmp/dovecot-deliver-errors.log
info_log_path = /tmp/dovecot-deliver.log
}

The original Evolution filters rules were stored in $HOME/.config/evolution/mail/filters.xml, so I wrote a quick python DOM parsing tool to extract the (long) list of definitions and generate Sieve rules for the $HOME/.dovecot.sieve config file (in which folder hierarchy uses a ‘.’ path separator instead of a ‘/’).

The sieve-test program can be used to check is sieve rules function as you want.

Now this is all setup, I only need to configure notmuch to index all the Maildirs. But this is left to another post.

Migrating mail from Evolution local storage to Evolution + local Imap (dovecot with Maildir) 1/X

I’ve been fed up with Evolution being so buggy and slow, which was making managing my mail ever more painful as time passes. Still, one of the advantages of Evolution over other MUAs is the vfolders feature, which I’ve been using intensively for years. Would there be some alternate way of managing my mail ?

I’ve been reading about notmuch‘s progress in becoming a very powerful mail management / retrieval tool (it could very well replace the vfolder feature of Evolution), but unfortunately, it is not (yet) integrated with Evolution.

Others have documented their mail setup, and it seems that I could achieve a middle ground solution by migrating all my mail folders, currently stored inside Evolution’s local storage, into Maildir folders managed by a local dovecot IMAP server. Evolution would still be my MUA, but it would store my mail inside Maildirs through dovecot. I’ve been told that Evolution may be less buggy with IMAP than with local folders, which could solve my main annoyances. Also, it would make it possible to index mail with notmuch, as it supports the Maildir storage format.

I’ve then decided to take the opportunity of a holiday period to start the migration attempt.

I’ll document it in another post.

Migrating from Evolution to Gnus + NotMuch (part 1)

I’ve gotten fed up with Evolution‘s bugs and slowness. Even though I experience less problems than some weeks ago since I’ve applied a patch related to locks in e-d-s, Evolution has now become really too slow… and given that my mail flow is not really diminishing, I’ve come to the point when I think about migrating.

I’ve been a user of Gnus for years also (at home, with much satisfaction, as it runs fine inside a GNU Screen), so I could switch to it with moderate damage. Also, as I’m using org-mode, it would be quite complementary of course, staying more in Emacs 😉

One nice (and even priceless) feature of Evolution is the vfolders, that allow me to manage the tons of emails in different contexts, wherever that may be located. AFAIK, Gnus doesn’t offer any comparable feature. But it seems that the most interesting way to have it is to integrate Gnus with NotMuch. Fortunately, my colleague Roland has written a nice howto explaining (among other details of his mail system setup) how to integrate these together.

Now, if I’m able to setup a similar NotMuch + Gnus setup for newly received mail, I’m left with migrating all my piles of old mail, currently stored in Evolution.

It seems that one possible way to do so is through copying Evolution mail folders into a newly setup local IMAP server’s Maildir storage. The copy is supposed to preserve some flags like the read/unread status of the “important” flag.

From the first tests I’ve made, it’s possible to install a local Dovecot IMAPd server, configure it so that the mail is stored in the user’s ~/Maildir/ maildir directory, which can then be recognized by Evolution as a target for copying mail. It seems that nomuch can then be configured so that it doesn’t flag all new mails as unread, and understangs the imap server flags accordingly (unread, important, and such). Needless to say, I’m glad all these useful programs are packaged in Debian 😉

I need to make further tests and also test Gnus + Notmuch integration, but having a possible solution to migrate my existing mail looks like a relief.

I’d be curious to read your alternative ideas for such a migration.

Stay tuned for next iterations.