SimpleQueue.get()'s body attribute is a string when using py-amqp, but a buffer with librabbitmq

It’s my understanding that librabbitmq is meant to be a drop-in replacement for py-amqp, that kombu will use if present and will otherwise fall back to the latter.

Using Python 2.7.11, Kombu 3.0.32 & amqp 1.4.8, I’ve just tried adding librabbitmq 1.6.1 to our requirements file in mozilla/treeherder#1225, however now get test failures where there were none before: https://travis-ci.org/mozilla/treeherder/jobs/100121889#L452

=============== FAILURES ===============
____ test_job_retrigger_authorized _____

webapp = <webtest.app.TestApp object at 0x7fa9d6db18d0>
eleven_jobs_stored = None
jm = <treeherder.model.derived.jobs.JobsModel object at 0x7fa9d44d4b50>
pulse_action_consumer = <kombu.simple.SimpleQueue object at 0x7fa9d4519f50>

    def test_job_retrigger_authorized(webapp, eleven_jobs_stored, jm,
                                      pulse_action_consumer):
        """
        Validate that only authenticated users can hit this endpoint.
        """
        client = APIClient()
        email = "[e-mail]"
        user = User.objects.create(username="retrigger-fail", email=email)
        client.force_authenticate(user=user)

        job = jm.get_job_list(0, 1)[0]
        job_id_list = [job["id"]]
        url = reverse("jobs-retrigger",
                      kwargs={"project": jm.project})
        client.post(url, {"job_id_list": job_id_list}, format='json')

        message = pulse_action_consumer.get(block=True, timeout=2)
>       content = json.loads(message.body)

tests/webapp/api/test_jobs_api.py:148: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/opt/python/2.7.11/lib/python2.7/json/__init__.py:339: in loads
    return _default_decoder.decode(s)
/opt/python/2.7.11/lib/python2.7/json/decoder.py:364: in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <json.decoder.JSONDecoder object at 0x7fa9dd350050>
s = <read-only buffer ptr 0x3f35557, size 216 at 0x7fa9d44b1ef0>
idx = 0
    def raw_decode(self, s, idx=0):
        """Decode a JSON document from ``s`` (a ``str`` or ``unicode``
            beginning with a JSON document) and return a 2-tuple of the Python
            representation and the index in ``s`` where the document ended.

            This can be used to decode a JSON document from a string that may
            have extraneous data at the end.

            """
        try:
>           obj, end = self.scan_once(s, idx)
E           TypeError: first argument must be a string, not buffer

/opt/python/2.7.11/lib/python2.7/json/decoder.py:380: TypeError

It seems like the APIs for py-amqp and librabbitmq are not identical after all, and maybe kombu needs to handle this somehow? (Or alternatively there’s a bug in amqp or librabbitmq.)

For our test run, we configure Kombu here: https://github.com/mozilla/treeherder/blob/985cd86a6de3b33a5025b7152be4f29804456eba/tests/conftest.py#L324

And the failure test is here: https://github.com/mozilla/treeherder/blob/5bdc6ab6a8cd14e1d3907a4310bc4f5bebf9048f/tests/webapp/api/test_jobs_api.py#L131

Any ideas? Thanks 😃

[ask]

.body is the raw message body, .payload is the deserialized payload that handles all of this stuff for you

[edmorley]

Performing a .get() on the SimpleQueue object was returning an object whose body attribute was a string when using py-amqp, which we were then deserializing manually until now. When using librabbitmq, the body attribute is instead of type buffer.

As a user, it’s not expected that the behaviour should vary like this depending on which rabbitmq library is used, particularly when the Kombu docs imply that librabbitmq should just be a drop-in replacement.

Many thanks 😃