IMAP Consumption - Mails imported multiple times and not marked as read or deleted

Info

I am currently using the docker image provided by https://github.com/linuxserver/docker-papermerge. However, I guess it is a logical bug not based on docker itself. Papermerge is running on my Raspberry Pi 4.

Expected Behavior

Using the IMPORT_MAIL_DELETE option in papermerge.conf.py should delete processed emails from the IMAP account and the email should be imported once into the inbox of the superuser papermerge account. Further, the mail should be marked as read if not deleted … or how does the papermerge workers differentiate new vs. old mails?

Current Behavior

The email is successfully processed by the worker and imported into the papermerge web application. However, emails are not marked as read or deleted. This leads to an issue, where the same email is being imported over and over again into the inbox folder of papermerge’s web application.

Steps to Reproduce

1. Configure papermerge.conf.py to process emails
2. Send an email with a PDF attachment to the IMAP account watched by papermerge’s worker process
3. Log into the papermerge web application as superuser
4. Observe that the same email is processed multiple times every 30s and imported over and over again. Further, the email won’t be deleted or marked as read from the mailserver. Therefore, the inbox gets overwhelmed by the same email over time.

Environment

Raspberry Pi 4 - ARM Linux omv 4.19.75-v7l+ #1271 armv7l GNU/Linux Docker version 19.03.13, build 4484c46 linuxserver/papermerge:latest

[ciur]

Now fix is in Papermerge Core 2.0.0rc48.

During next couple of days I will:

• switch this repository CI to github actions (now it is using Travis CI, which is obsolete)
• automate building and publishing of docker image with github actions
• tag next release iteration (which will trigger building and publishing of docker image based on release tag)

It will take some time, but finally will be done right. (now I manually build and publish docker image, which is a pain)

[ciur]

@l4rm4nd, oh, man, I am impressed 👍 . I didn’t know about server.add_flags(uid, br'\Seen')! Thank you for the tip! I will review/test/have a look at the issue in couple of days. Right now, I am recording screencasts and writing blog posts about Paparmerge features. If you want (maybe you need fix urgently, like today 😃 ), you may create a PR with the changes.

In 2.0 the whole IMAP code was refactored taking into account import by user and import by secret features; imho now code in papermerge.core.importers.imap module is cleaner and easier to read. The entry point for importing document attachments is here, which as you can see is part of different repository (papermerge-core is a reusable django app).

I like you security acumen 🥇 , maybe you may wear, for a day or two, developer hat 🎩 as well 😃

Also, you might be interested in new (again in 2.0, thus it is in papermerge-core repository) management command called imap, I wrote it to quickly troubleshoot IMAP related issues. You use it like any standard django command:

$ ./manage.py  imap --connect # checks if can connect to IMAP account
$ ./manage.py  imap --import    # imports the documents from newly (unread) emails