Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Sync error when encountering UTF-8 data in BODYSTRUCTURE response

See original GitHub issue

Checklist

I have used the search function to see if someone else has already submitted the same bug report.
I will describe the problem with as much detail as possible.

App version

6.301

Where did you get the app from?

Google Play

Android version

Device model

Mi A3

Steps to reproduce

Synchronize the “INBOX” folder

Expected behavior

Synchronize the “INBOX” folder with success

Actual behavior

Does not synchronize the folder, failed silently, traceback in logs

Logs

09-10 09:26:33.894 22121 22144 D ImapSync: SYNC: Synced unsynced messages for folder INBOX
09-10 09:26:33.895 22121 22144 D ImapSync: SYNC: Have 12 large messages and 0 small messages out of 12 unsynced messages
09-10 09:26:33.895 22121 22144 D ImapSync: SYNC: Fetching 0 small messages for folder INBOX
09-10 09:26:33.895 22121 22144 D ImapSync: SYNC: Done fetching small messages for folder INBOX
09-10 09:26:33.895 22121 22144 D ImapSync: SYNC: Fetching large messages for folder INBOX
09-10 09:26:33.896 22121 22144 V RealImapConnection: conn151459594>>> 13 UID FETCH 1896,1898,1900,1902,1905,1906,1907,1908,1909,1911,1913,1915 (UID BODYSTRUCTURE)
09-10 09:26:33.910 22121 22144 V RealImapConnection: conn151459594<<<#null# [1535, FETCH, [UID, 1896, BODYSTRUCTURE, [[[TEXT, PLAIN, [CHARSET, utf-8], NIL, NIL, QUOTED-PRINTABLE, 11050, 148, NIL, NIL, NIL, NIL], [TEXT, HTML, [CHARSET, utf-8], NIL, NIL, QUOTED-PRINTABLE, 21720, 384, NIL, NIL, NIL, NIL], ALTERNATIVE, [BOUNDARY, <REMOVE>], NIL, NIL, NIL], [APPLICATION, PDF, [NAME, Ing��nieur ��tude et d��veloppement.pdf], NIL, NIL, BASE64, 114774, NIL, [ATTACHMENT, [FILENAME, Ing��nieur ��tude et d��veloppement.pdf]], NIL, NIL], MIXED, [BOUNDARY, <REMOVE>], NIL, NIL, NIL]]]
09-10 09:26:33.911 22121 22144 E ImapSync: synchronizeMailbox
09-10 09:26:33.911 22121 22144 E ImapSync: java.lang.IllegalArgumentException: Unsupported character: �
09-10 09:26:33.911 22121 22144 E ImapSync: 	at com.fsck.k9.mail.internet.MimeParameterEncoder.quoted(MimeParameterEncoder.kt:161)
09-10 09:26:33.911 22121 22144 E ImapSync: 	at com.fsck.k9.mail.store.imap.RealImapFolder.parseBodyStructure(RealImapFolder.kt:894)
09-10 09:26:33.911 22121 22144 E ImapSync: 	at com.fsck.k9.mail.store.imap.RealImapFolder.parseBodyStructure(RealImapFolder.kt:830)
09-10 09:26:33.911 22121 22144 E ImapSync: 	at com.fsck.k9.mail.store.imap.RealImapFolder.handleFetchResponse(RealImapFolder.kt:748)
09-10 09:26:33.911 22121 22144 E ImapSync: 	at com.fsck.k9.mail.store.imap.RealImapFolder.fetch(RealImapFolder.kt:599)
09-10 09:26:33.911 22121 22144 E ImapSync: 	at com.fsck.k9.backend.imap.ImapSync.downloadLargeMessages(ImapSync.kt:561)
09-10 09:26:33.911 22121 22144 E ImapSync: 	at com.fsck.k9.backend.imap.ImapSync.downloadMessages(ImapSync.kt:367)
09-10 09:26:33.911 22121 22144 E ImapSync: 	at com.fsck.k9.backend.imap.ImapSync.synchronizeMailboxSynchronous(ImapSync.kt:198)
09-10 09:26:33.911 22121 22144 E ImapSync: 	at com.fsck.k9.backend.imap.ImapSync.sync(ImapSync.kt:34)
09-10 09:26:33.911 22121 22144 E ImapSync: 	at com.fsck.k9.backend.imap.ImapBackend.sync(ImapBackend.kt:55)
09-10 09:26:33.911 22121 22144 E ImapSync: 	at com.fsck.k9.controller.MessagingController.syncFolder(MessagingController.java:644)
09-10 09:26:33.911 22121 22144 E ImapSync: 	at com.fsck.k9.controller.MessagingController.synchronizeMailboxSynchronous(MessagingController.java:584)
09-10 09:26:33.911 22121 22144 E ImapSync: 	at com.fsck.k9.controller.MessagingController.lambda$synchronizeMailbox$3(MessagingController.java:556)
09-10 09:26:33.911 22121 22144 E ImapSync: 	at com.fsck.k9.controller.MessagingController.$r8$lambda$ar_6eNiqt1c67rfo1nCyGoIX3Sg(Unknown Source:0)
09-10 09:26:33.911 22121 22144 E ImapSync: 	at com.fsck.k9.controller.MessagingController$$ExternalSyntheticLambda20.run(Unknown Source:10)
09-10 09:26:33.911 22121 22144 E ImapSync: 	at com.fsck.k9.controller.MessagingController.runInBackground(MessagingController.java:223)
09-10 09:26:33.911 22121 22144 E ImapSync: 	at com.fsck.k9.controller.MessagingController.access$000(MessagingController.java:109)
09-10 09:26:33.911 22121 22144 E ImapSync: 	at com.fsck.k9.controller.MessagingController$1.run(MessagingController.java:164)
09-10 09:26:33.911 22121 22144 E ImapSync: 	at java.lang.Thread.run(Thread.java:923)
09-10 09:26:33.924 22121 22144 E ImapSync: Failed synchronizing folder <REMOVE>:INBOX @ Sat Sep 10 09:26:33 GMT+02:00 2022
09-10 09:26:33.924 22121 22144 I MessagingController:  Command 'synchronizeMailbox' completed
09-10 09:26:33.924 22121 22144 I MessagingController: Running command 'sendPendingMessages', seq = 46 (background priority)
09-10 09:26:33.926 22121 22144 I MessagingController:  Command 'sendPendingMessages' completed

Issue Analytics

State:
Created a year ago
Comments:8 (3 by maintainers)

Top GitHub Comments

1reaction

xaxcommented, Sep 13, 2022

For this bug, it doesn’t make much difference though. We need to change the IMAP code to handle non-ASCII values in a BODYSTRUCTURE response.

What adds to the severity to the current K9 behavior is, that (modulo date span filters) as long as a large message like the (offending?) one above is in a folder of an (offending‽) IMAP server, no large message in K9’s download/sync pipeline after it will be successfully downloaded and thus will never appear in the K9 IMAP folder view.

0reactions

cketticommented, Sep 12, 2022

RFC 6532 - Internationalized Email Headers only allows UTF-8 in email headers. So if it’s anything but that, it’s definitely an invalid encoding and the software that the sender is using should be fixed.

For this bug, it doesn’t make much difference though. We need to change the IMAP code to handle non-ASCII values in a BODYSTRUCTURE response. In my reading of RFC 3501 - INTERNET MESSAGE ACCESS PROTOCOL - VERSION 4rev1 it’s a protocol violation (“Characters are 7-bit US-ASCII unless otherwise specified”). However, it’s an easy mistake to make. And with UTF-8 in message headers becoming more common, this will happen more often. So we might as well work around this server bug.

Top Results From Across the Web

How can I fix the UTF-8 error when bulk uploading users?

Answer · Open your CSV file in Microsoft Excel, then click File > Save As · Enter any name for the file, then...

RFC 3501: INTERNET MESSAGE ACCESS PROTOCOL

Obsolete commands, responses, and data formats which an IMAP4rev1 ... synchronization errors; see the description of the EXPUNGE response for more detail.

fetch / response.text() messes up the utf-8 - Stack Overflow

I am currently fetching(https://data.

FAQ - UTF-8, UTF-16, UTF-32 & BOM - Unicode

The downside of UTF-8 is that without converting into a format that can be displayed on your system, you cannot tell which non-ASCII...

Invalid UTF8 detected in string - Snowflake Community

Reply. seeling.cheung (Snowflake). 5 years ago. Hello. Snowflake supports UTF8 encoding only. If you get this error, it's likely that your data were...