ImapProtocolException: Syntax error in BODYSTRUCTURE. Unexpected token: NIL" w/ Office365

Describe the bug I have an MVC5 application which is using IMAP to load messages from Office365. The code lists the UniqueId and InternalDate of the messages in the inbox, then uses FetchAsync to fetch the following properties for the loaded message IDs:

• UniqueId;
• Flags;
• Envelope;
• PreviewText;
• BodyStructure;
• The X-Priority header;

We have received a message which causes this method to throw a TaskCancelledException. This happens even if we pass in CancellationToken.None, so it’s not our code that’s cancelling the operation.

Platform (please complete the following information):

• OS: Windows (reproduced on 11 and Server 2019)
• .NET Runtime: .NET Framework
• .NET Framework: .NET 4.8
• MailKit Version: 3.3.0

Exception

System.Threading.Tasks.TaskCanceledException: A task was canceled.
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at MailKit.Net.Imap.ImapEngine.<ProcessUntaggedResponseAsync>d__189.MoveNext() in D:\src\MailKit\MailKit\Net\Imap\ImapEngine.cs:line 2234
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at MailKit.Net.Imap.ImapCommand.<StepAsync>d__84.MoveNext() in D:\src\MailKit\MailKit\Net\Imap\ImapCommand.cs:line 915
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at MailKit.Net.Imap.ImapEngine.<IterateAsync>d__190.MoveNext() in D:\src\MailKit\MailKit\Net\Imap\ImapEngine.cs:line 2345
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at MailKit.Net.Imap.ImapEngine.<RunAsync>d__191.MoveNext() in D:\src\MailKit\MailKit\Net\Imap\ImapEngine.cs:line 2366
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at MailKit.Net.Imap.ImapFolder.<FetchAsync>d__193.MoveNext() in D:\src\MailKit\MailKit\Net\Imap\ImapFolderFetch.cs:line 1048
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
  at (my code...)

Protocol Logs

Connected to imaps://outlook.office365.com:993/ S: * OK The Microsoft Exchange IMAP4 service is ready. [TABPADYAUAAxADIAMwBDAEEAMAAwADMANwAuAEcAQgBSAFAAMQAyADMALgBQAFIATwBEAC4ATwBVAFQATABPAE8ASwAuAEMATwBNAA==] C: B00000000 CAPABILITY S: * CAPABILITY IMAP4 IMAP4rev1 AUTH=PLAIN AUTH=XOAUTH2 SASL-IR UIDPLUS ID UNSELECT CHILDREN IDLE NAMESPACE LITERAL+ S: B00000000 OK CAPABILITY completed. C: B00000001 AUTHENTICATE PLAIN ******** S: B00000001 NO AUTHENTICATE failed. C: B00000002 LOGIN “" "” S: B00000002 OK LOGIN completed. C: B00000003 CAPABILITY S: * CAPABILITY IMAP4 IMAP4rev1 AUTH=PLAIN AUTH=XOAUTH2 SASL-IR UIDPLUS MOVE ID UNSELECT CLIENTACCESSRULES CLIENTNETWORKPRESENCELOCATION BACKENDAUTHENTICATE CHILDREN IDLE NAMESPACE LITERAL+ S: B00000003 OK CAPABILITY completed. C: B00000004 NAMESPACE S: * NAMESPACE ((“” “/”)) NIL NIL S: B00000004 OK NAMESPACE completed. C: B00000005 LIST “” “INBOX” S: * LIST (\Marked \HasChildren) “/” INBOX S: B00000005 OK LIST completed. C: B00000006 LIST “” INBOX/Ignore S: * LIST (\Marked \HasNoChildren) “/” INBOX/Ignore S: B00000006 OK LIST completed. C: B00000007 EXAMINE INBOX/Ignore S: * 1 EXISTS S: * 1 RECENT S: * FLAGS (\Seen \Answered \Flagged \Deleted \Draft $MDNSent) S: * OK [PERMANENTFLAGS ()] Permanent flags S: * OK [UIDVALIDITY 4340] UIDVALIDITY value S: * OK [UIDNEXT 3] The next unique identifier value S: B00000007 OK [READ-ONLY] EXAMINE completed. C: B00000008 UID FETCH 2 (UID FLAGS ENVELOPE BODYSTRUCTURE BODY.PEEK[HEADER.FIELDS (X-PRIORITY)]) S: * 1 FETCH (UID 2 FLAGS (\Seen \Recent) ENVELOPE (“Tue, 2 Aug 2022 15:01:06 +0000” “Returned mail: see transcript for details” ((“Mail Delivery Subsystem” NIL “MAILER-DAEMON” “hermes.gatewaynet.com”)) NIL NIL ((NIL NIL “MAILBOX” “OUR-DOMAIN”)) NIL NIL NIL “<202208021501.272F16D4031920@HERMES-GATEWAYNET-COM>”) BODYSTRUCTURE ((NIL NIL NIL NIL NIL “7BIT” 563 NIL NIL NIL NIL)(“message” “delivery-status” NIL NIL NIL “7BIT” 658 NIL NIL NIL NIL)(“message” “rfc822” NIL NIL NIL “8bit” 0 (“Tue, 2 Aug 2022 15:00:47 +0000” “[POSSIBLE SPAM 11.4] Invoices now overdue - 115365#” ((NIL NIL “MAILBOX” “OUR-DOMAIN”)) NIL NIL ((NIL NIL “accounts” “OTHER-DOMAIN”) (NIL NIL “safety” “OTHER-DOMAIN”) (NIL NIL “USER” “OUR-DOMAIN”)) NIL NIL NIL “<1IOGPFNLIHU4.377MHPZYJQ6E3@OUR-SERVER>”) (((“text” “plain” (“charset” “utf-8”) NIL NIL “8bit” 597 16 NIL NIL NIL NIL)((“text” “html” (“charset” “utf-8”) NIL NIL “7BIT” 1611 26 NIL NIL NIL NIL)(“image” “png” (“name” “0.dat”) “<1KWGPFNLIHU4.4RR7HCVM8MQQ1@OUR-SERVER>” NIL “base64” 14172 NIL (“inline” (“filename” “0.dat”)) NIL “0.dat”)(“image” “png” (“name” “1.dat”) “<1KWGPFNLIHU4.UWJ8R86RE2KA2@OUR-SERVER>” NIL “base64” 486 NIL (“inline” (“filename” “1.dat”)) NIL “1.dat”)(“image” “png” (“name” “2.dat”) “<1KWGPFNLIHU4.EC7HN124OJC32@OUR-SERVER>” NIL “base64” 506 NIL (“inline” (“filename” “2.dat”)) NIL “2.dat”)(“image” “png” (“name” “3.dat”) “<1KWGPFNLIHU4.WM1ALJTG745F1@OUR-SERVER>” NIL “base64” 616 NIL (“inline” (“filename” “3.dat”)) NIL “3.dat”)(“image” “png” (“name” “4.dat”) “<1KWGPFNLIHU4.1B42S5EVSF4B2@OUR-SERVER>” NIL “base64” 22470 NIL (“inline” (“filename” “4.dat”)) NIL “4.dat”) “related” (“boundary” “=-5nEE2FIlRoeXkJyZAHV8UA==” “type” “text/html”) NIL NIL) “alternative” (“boundary” “=-1sRjeMizXVbc5nGIFXbARA==”) NIL NIL)(“application” “pdf” (“name” “Reminder.pdf”) “<RJ2DSFNLIHU4.UUVSNNY5Z3ER@OUR-SERVER>” NIL “base64” 359650 NIL (“attachment” (“filename” “Reminder.pdf” “size” “262820”)) NIL NIL) “mixed” (“boundary” “=-EJwVTfPtacyNnTqY4DPQ0A==”) NIL NIL) 0 NIL NIL NIL NIL) “report” (“report-type” “delivery-status” “boundary” “272F16D4031920.1659452466/hermes.gatewaynet.com”) NIL NIL) BODY[HEADER.FIELDS (X-PRIORITY)] {2} S: S: )

Additional context After moving the affected message out of the folder, the FetchAsync command works as expected.

Using GetMessageAsync, I am able to load the affected message without any problems.

[jstedfast]

3.4.1.515 is a simplified version of the fix.

[jstedfast]

Yea, probably worth changing.

Anyway, it’s times like this that I’m thankful that I added unit tests for all of the workarounds for all of the other breakages. haha.

I have a fix locally for this particular issue, but it broke other work-arounds. That’s where I’m having to rethink how to go about all of these work-arounds for the various breakages I’ve added work-arounds for.

This particular one throws a wrench in the works because of all the NILs.