[Mairix-users] IMAP support in mairix

[Kim Vandry]

Hello mairix users,

I have been using mairix as the primary tool for searching my mailbox
for many years and I have been very happy with it.

Following a Dovecot upgrade on the server where my mail is located, my
whole mailbox will soon be in Dovecot's proprietary dbox format, which
of course mairix does not support (my mailbox is presently in maildir
format). I would hate to give up mairix so...

I have added an IMAP client to mairix so that it supports indexing and
searching messages found on an IMAP server and placing search results in
a folder on the same IMAP server. Both features are optional, so that
you can also index remote messages and store the results locally, or
index local folders and store the results on IMAP.

My changes are made up of about 8 patches.

I would like to know if you would be interested in accepting this
feature upstream. Please let me know.

Thanks

-kv

[Janusz S. Bien]

Post by Kim Vandry
Hello mairix users,
I have been using mairix as the primary tool for searching my mailbox
for many years and I have been very happy with it.
Following a Dovecot upgrade on the server where my mail is located, my
whole mailbox will soon be in Dovecot's proprietary dbox format, which
of course mairix does not support (my mailbox is presently in maildir
format). I would hate to give up mairix so...
I have added an IMAP client to mairix so that it supports indexing and
searching messages found on an IMAP server and placing search results in
a folder on the same IMAP server. Both features are optional, so that
you can also index remote messages and store the results locally, or
index local folders and store the results on IMAP.
My changes are made up of about 8 patches.
I would like to know if you would be interested in accepting this
feature upstream. Please let me know.

Sounds great! I hope it will be included upstream, but I am willing to
give it a try anyway.

Best regards

Janusz

[Edward Burns]

KV> Hello mairix users,
KV> I have been using mairix as the primary tool for searching my mailbox
KV> for many years and I have been very happy with it.

KV> Following a Dovecot upgrade on the server where my mail is located, my
KV> whole mailbox will soon be in Dovecot's proprietary dbox format, which
KV> of course mairix does not support (my mailbox is presently in maildir
KV> format). I would hate to give up mairix so...

KV> I have added an IMAP client to mairix so that it supports indexing and
KV> searching messages found on an IMAP server and placing search results in
KV> a folder on the same IMAP server. Both features are optional, so that
KV> you can also index remote messages and store the results locally, or
KV> index local folders and store the results on IMAP.

KV> My changes are made up of about 8 patches.

KV> I would like to know if you would be interested in accepting this
KV> feature upstream. Please let me know.

I would love this. Mairix is currently the only thing keeping me from
moving entirely to IMAP. Currently I am using fetchmail just because I
want the mail in mbox format for mairix.

Ed

[Kim Vandry]

Post by Kim Vandry
I have added an IMAP client to mairix so that it supports indexing and
searching messages found on an IMAP server and placing search results in
a folder on the same IMAP server. Both features are optional, so that
you can also index remote messages and store the results locally, or
index local folders and store the results on IMAP.

My changes for the IMAP feature can be found here:

https://github.com/vandry/mairix

Here are some brief instructions on how to use it.

Previously you had three ways of specifying the source folders in the
mairixrc file:

maildir=folder1:folder2
mh=folder1:folder2
mbox=folder1:folder2

Now you have a fourth option:

imap=folder1:folder2

That's for source folders. If you want to place the search results in an
IMAP folder too (recommended), you can do it like this:

mformat=imap

Of course you need to tell mairix how to connect to the IMAP server:

imap_server=mail.example.com

And the IMAP server probably needs for you to log in too, so this:

imap_username=myusername
imap_password=mypassword

If you have shell access to the same system that runs your IMAP server,
you may be able to connect directly to the IMAP server without using a
TCP/IP connection and it will know who you are so you don't have to give
it a username and password. In that case, you can use the following
option instead of imap_server, imap_username, and imap_password:

imap_pipe=/usr/lib/dovecot/imap


If you want to switch all your source folders from non-IMAP to IMAP (or
vice versa), you might want to consider deleting mairix's database file
and starting over from scratch because mairix will have to reindex all
of your mail anyway.

I have tested all of this with my own email and it seems to work so far.
Please let me know if you try it out and encounter any problems.

Happy weekend,

-kv

[Janusz S. Bien]

Quote/Cytat - Kim Vandry <***@TZoNE.ORG> (Fri 05 Sep 2014 09:12:15
PM CEST):
[...]

Post by Kim Vandry
I have tested all of this with my own email and it seems to work so far.
Please let me know if you try it out and encounter any problems.

I've just tested it and also it seems to work.

The users od Debian and derivates may be reminded of checkinstall:

https://wiki.debian.org/CheckInstall

Best regards

Janusz

[Janusz S. Bień]

Post by Kim Vandry

Post by Kim Vandry
I have added an IMAP client to mairix so that it supports indexing and
searching messages found on an IMAP server and placing search results in
a folder on the same IMAP server. Both features are optional, so that
you can also index remote messages and store the results locally, or
index local folders and store the results on IMAP.

https://github.com/vandry/mairix

I've used it without problems for months, but recently I've bought my
own domain and got my own IMAP server. The mail works OK on my Android
smartfon but running mairix results in the fatal error:

IMAP client cannot find capabilities list

I would appreciate very much your suggestions how to diagnose and solve
the problem.

Best regards

Janusz

[Kim Vandry]

Post by Janusz S. Bień

Post by Kim Vandry
https://github.com/vandry/mairix

I've used it without problems for months, but recently I've bought my
own domain and got my own IMAP server. The mail works OK on my Android
IMAP client cannot find capabilities list
I would appreciate very much your suggestions how to diagnose and solve
the problem.

Hello Janusz,

Finding the capability list from the IMAP server's greeting is one of
the first things the IMAP client does after connecting. There may be a
problem with the connection to the IMAP server or with the greeting it
sends. Try connecting to your IMAP server manually from the command line
to see what the greeting looks like. Use, for example, "telnet
<imap-server-name> 143" or "nc <imap-server-name> 143". The greeting
should include a cpability list. Here's my server's greeting for comparison:

* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE
IDLE STARTTLS LOGINDISABLED] Dovecot ready.

If the capabilities list includes "STARTTLS" then you can also test
encrypted connections to the IMAP server because that's what the mairix
IMAP client will do. Use this command:

openssl s_client -CApath /etc/ssl/certs -host <imap-server-name> -port
143 -starttls imap

Look for something like this in the last few lines of the output,
especially the "ok" verify return code:

Verify return code: 0 (ok)
---
. OK Pre-login capabilities listed, post-login capabilities have more.

-kv

------------------------------------------------------------------------------

[Janusz S. Bien]

Thank you very much for your quick response.

Quote/Cytat - Kim Vandry <***@TZoNE.ORG> (Thu 17 Dec 2015 02:54:18 AM CET):

[...]

Post by Kim Vandry
Hello Janusz,
Finding the capability list from the IMAP server's greeting is one of
the first things the IMAP client does after connecting. There may be a
problem with the connection to the IMAP server or with the greeting it
sends. Try connecting to your IMAP server manually from the command line
to see what the greeting looks like. Use, for example, "telnet
<imap-server-name> 143" or "nc <imap-server-name> 143". The greeting
* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE
IDLE STARTTLS LOGINDISABLED] Dovecot ready.

I get only

* OK domains020.home.pl IdeaImapServer v0.80 ready

Is there anything I can do except complaining to the provider?

Best regards

Janusz

[Janusz S. Bien]

Post by Janusz S. Bien
Thank you very much for your quick response.

[...]

Post by Janusz S. Bien
I get only
* OK domains020.home.pl IdeaImapServer v0.80 ready
Is there anything I can do except complaining to the provider?

Following

https://support.microsoft.com/en-us/kb/189325

I tried

abcd CAPABILITY

and got

* CAPABILITY IMAP4rev1 LITERAL+ CHILDREN I18NLEVEL=1 IDLE SORT UIDPLUS
UNSELECT XLIST STARTTLS AUTH=PLAIN AUTH=LOGIN
abcd OK Completed

So perhaps mairix can do the same?

BTW, what "abcd" means in the example?

Best regards

Janusz

[Kim Vandry]

Post by Janusz S. Bien
I get only
* OK domains020.home.pl IdeaImapServer v0.80 ready

Thank you, Janusz, that's what I needed to know.

Your IMAP server does not send its capability list in the initial
greeting. Clients have to ask for it specifically (as you demonstrated
in your next email). There's nothing wrong with your server: the IMAP
RFC does not require servers to send the capability list in the initial
greeting. Unfortunately my code assumed that the server does do it, so
it doesn't work.

I have modified the code so that it supports this kind of server.

https://github.com/vandry/mairix/commit/15777bd1fdcaf92c2c7c376c253fe26cc5f788fb

-kv

------------------------------------------------------------------------------

[Janusz S. Bien]

Quote/Cytat - Kim Vandry <***@TZoNE.ORG> (Thu 17 Dec 2015 03:03:48 PM CET):

[...]

Post by Kim Vandry
I have modified the code so that it supports this kind of server.
https://github.com/vandry/mairix/commit/15777bd1fdcaf92c2c7c376c253fe26cc5f788fb

Thank you very much!

I have now another problem:

STARTTLS: server name "zjb.waw.pl" != certificate common name "*.home.pl"

Any suggestions?

Best regards

Janusz

[Kim Vandry]

Hi Janusz,
We're making progress!

Post by Janusz S. Bien
STARTTLS: server name "zjb.waw.pl" != certificate common name "*.home.pl"

That seems like a legitimate error. It's the same error you get if you
connect to a secure web site using your web browser and the security
certificate does not match the host name. In this case, your have a
wildcard certificate that is valid for any server name ending in
"home.pl" but your IMAP server's name is in a different domain.

Does your IMAP server have an alternative host name (alias) that is
<something>.home.pl? If yes, you should be able to connect to it with
that host name and the certificate name will match and the connection
will proceed.

-kv

------------------------------------------------------------------------------

[Janusz S. Bien]

Post by Kim Vandry
Hi Janusz,
We're making progress!

Yes!

Post by Kim Vandry

Post by Janusz S. Bien
STARTTLS: server name "zjb.waw.pl" != certificate common name "*.home.pl"

[...]

Post by Kim Vandry
Does your IMAP server have an alternative host name (alias) that is
<something>.home.pl?

Yes, it was even in the FAQ.

However now I cannot log in:

Finding all currently existing messages...
IMAP authentication failed

The same problem when I use telnet:

telnet serwer123456.home.pl 143
...
a1 NO Login incorrect

Following http://www.anta.net/misc/telnet-troubleshooting/imap.shtml

I tried also

openssl s_client -connect serwer123456.home.pl:993 -quiet

with the same outcome.

Thi time I've reported the problem to the provider and i'm waitin for
an answer.

Best regards

Janusz

[Janusz S. Bień]

On Fri, Dec 18 2015 at 7:20 CET, ***@mimuw.edu.pl writes:

[...]

Post by Janusz S. Bien
Following http://www.anta.net/misc/telnet-troubleshooting/imap.shtml
I tried also
openssl s_client -connect serwer123456.home.pl:993 -quiet
with the same outcome.
Thi time I've reported the problem to the provider and i'm waitin for
an answer.

It appeared that I have to use a different server address, but it was
only one aspect of the problem. I was able to solve it thanks to the
kind help of Kim Vandry off the list. I will quote him for the archive:

On Fri, Dec 18 2015 at 13:55 CET, ***@TZoNE.ORG writes:

[...]

Post by Janusz S. Bien
There are two ways to connect securely to IMAP servers: IMAPS and
STARTTLS. With IMAPS, the client makes an SSL connection directly to
port 993. With STARTTLS, the client makes an unencrypted connection to
port 143 (the usual, and original, port for IMAP) and then upgrades
the unencrypted connection to an encrypted one. Most servers support
both ways. In the mairix client, I only implemented STARTTLS.
openssl s_client -host serwer123456.home.pl -port 993
openssl s_client -host serwer123456.home.pl -port 143 -starttls imap
Assuming the server supports both IMAPS and STARTTLS, then if you try
either one of those test commands, you should be able to type IMAP
t1 CAPABILITY
t2 LOGIN "username" "password"
t3 LOGOUT

Best regards

Janusz