Dovecot and Argon2 doesn’t work? This may be why.

Sorry, title is too long. Again. Nevertheless, ignore this and continue on…

I run my own mailserver. I love doing this, don’t ask me why. Anyway, I was recently migrating a few password hashes to Argon2. I confirmed manually that everything was working, I checked that Dovecot was able to generate and verify Argon2 hashes, that my backend storage was doing everything correctly and so on.

Then I changed a bunch of passwords to migrate them over to Argon2 (I was previously using bcrypt, but I started to like Argon2 more because of it’s resistance to many optimization attacks). Just after I had those new Argon2 hashes in the database, I could no longer login using these users. I think it worked like once and then never again.

Well, damn. I spend hours researching what may be wrong. Dovecot was simply spitting out it’s usual “dovecot: auth-worker(pid): Password mismatch” message. Nothing I could get any information from. To summarize what I found in the ‘net: Nothing of use.

So well, why am I writing this post then? Well because I finally figured out what’s wrong. The Dovecot documentation states this:

ARGON2 can require quite a hefty amount of virtual memory, so we recommend that you set service auth { vsz_limit = 2G } at least, or more.

https://doc.dovecot.org/configuration_manual/authentication/password_schemes/

Well, I obviously already did that – I do read the documentation from time to time, at least when I’m trying to solve a critical problem. But you know what’s strange? The docs also state that the actual login isn’t handled by the auth service, instead it’s usually done by a service called auth-worker ¹[at least if you’re using a database like I do] (that’s also the thing that’s producing the “Password mismatch” log messages).

To make a long story short, what happened was that Dovecot stopped the auth-worker process as it was trying to hash the Argon2 password. This simply triggered a generic “Password mismatch” message instead of something useful like “out of memory”, so yeah…

Lesson Learned: If you’re using Argon2, increase the memory limit of auth-worker, not the auth service like the docs tell you to.

This was the solution. I simply added a few config lines to Dovecot:

service auth-worker {
   # Needed for argon2. THIS IS THE IMPORTANT SETTING!
   vsz_limit = 0 # Means unlimited, other values like 2G or more also also valid
   # Other custom settings for your auth-workers here...
}

And login was working again. I never found anyone mentioning that you need to set this value for auth-worker instead of auth, which is why I wrote this little post.

1. This varies from passdb and/or userdb type. Some types may use the auth service directly, others use an auth worker. Check what your setup is using before taking my words for granted.