error when pickle ServiceBusReceivedMessage object

• Package Name: azure-servicebus
• Package Version: 7.8.1
• Operating System: alpine 3.12
• Python Version: 3.8.6

Describe the bug There is an error when trying to pickle the ServiceBusReceivedMessage object. The “prefork” pool (multiprocessing) in celery uses pickle for serialization, and from v5.2 it is not working, see details in https://github.com/celery/celery/issues/7237

To Reproduce It can be reproduced by using the following code:

import os
import json
import pickle
from azure.servicebus import ServiceBusClient, ServiceBusMessage

CONNECTION_STR = "XXX"
QUEUE_NAME = "XXX"

msg = {
   "body":"W1tdLCB7ImlucHV0X2lkIjogNH0sIHsiY2FsbGJhY2tzIjogbnVsbCwgImVycmJhY2tzIjogbnVsbCwgImNoYWluIjogbnVsbCwgImNob3JkIjogbnVsbH1d",
   "content-encoding":"utf-8",
   "content-type":"application/json",
   "headers":{
      "lang":"py",
      "task":"tasks.example_task",
      "id":"7c66557d-e4bc-437f-b021-b66dcc39dfdf",
      "shadow":None,
      "eta":"2021-10-07T02:30:23.764066+00:00",
      "expires":None,
      "group":None,
      "group_index":None,
      "retries":1,
      "timelimit":[
         None,
         None
      ],
      "root_id":"7c66557d-e4bc-437f-b021-b66dcc39dfdf",
      "parent_id":"7c66557d-e4bc-437f-b021-b66dcc39dfdf",
      "argsrepr":"()",
      "kwargsrepr":"{'input_id': 4}",
      "origin":"gen36@94713e01a9c0",
      "ignore_result":1,
      "x_correlator":"44a1978d-c869-4173-afe4-da741f0edfb9"
   },
   "properties":{
      "correlation_id":"7c66557d-e4bc-437f-b021-b66dcc39dfdf",
      "reply_to":"7b9a3672-2fed-3e9b-8bfd-23ae2397d9ad",
      "origin":"gen68@c33d4eef123a",
      "delivery_mode":2,
      "delivery_info":{
         "exchange":"",
         "routing_key":"celery_task_queue"
      },
      "priority":0,
      "body_encoding":"base64",
      "delivery_tag":"dc83ddb6-8cdc-4413-b88a-06c56cbde90d"
   }
}

servicebus_client = ServiceBusClient.from_connection_string(conn_str="XXX")

with servicebus_client:
    with servicebus_client.get_queue_sender(QUEUE_NAME) as sender:
        # Sending a single message
        single_message = ServiceBusMessage(json.dumps(msg))
        sender.send_messages(single_message)


with servicebus_client:
    receiver = servicebus_client.get_queue_receiver(queue_name=QUEUE_NAME)
    with receiver:
        received_msgs = receiver.receive_messages(max_message_count=10, max_wait_time=5)
        for msg in received_msgs:
            print(str(msg))
            pickle.dumps(msg)
            receiver.complete_message(msg)

print("Receive is done.")

The result is:

Traceback (most recent call last):
  File "test_pickle.py", line 40, in <module>
    pickle.dumps(msg)
  File "stringsource", line 2, in uamqp.c_uamqp.CompositeValue.__reduce_cython__
TypeError: no default __reduce__ due to non-trivial __cinit__

Expected behavior It should works.

Screenshots If applicable, add screenshots to help explain your problem.

Additional context There are several bugs opened in celery and billiard which might be related to this issue: https://github.com/celery/celery/issues/7237 https://github.com/celery/billiard/issues/356

[IsmaelJS]

Hi @swathipil , I have updated the code within “To Reproduce” section with the data used. Thank you so much.

[kashifkhan]

Thank you the feedback @IsmaelJS. We will investigate and get back to you asap