Bytes and string both get encoded as strings

Hello community,

I have a problem when sending a byte array with the use of the msgpack serializer. When receiving the data, it is unpack as a string.

When using the json serializer, the data is correctly encoded, as referenced in this document, and decoded as bytes.

I also made Wireshark captures which show that the following byte array b'\x01\x02\x03\x04\x05\x06\x07\x08\x09\x10', is encoded as follows ["\u0000b'AQIDBAUGBwgJEA==' "], when using the json serializer. This encoding corresponds well to that described in the standard cited in the previous document. With the msgpack serializer, the byte array is not encoded correctly and send derecly as follow 0a 01 02 03 04 05 06 07 08 09 10

here is my current configuration:

• Python3.8
• autobahn 20.4.3
• msgpack 1.0.0

Below an example allowing to highlight the problem:

Backend.py

import asyncio
from autobahn.asyncio.wamp import ApplicationSession
from autobahn.asyncio.component import Component, run

class Backend(ApplicationSession):
    def __init__(self, config=None):
        config.realm = 'TESTS'
        super().__init__(config)


    async def onJoin(self, details):
        print('Register test_echo_payload')
        await self.register(self.test_echo_payload, 'test.echo.payload')

    async def onDisconnect(self):
        loop  = asyncio.get_event_loop()
        loop.stop()


    async def test_echo_payload(self, value: bytes) -> bytes:
        if not isinstance(value, bytes):
            print('Value is not an instance of bytes but is a %s' % (type(value)))
        return value


if __name__ == '__main__':
    backend = Component(
            session_factory = Backend,
            transports      = [
                {
                    'type':        'websocket',
                    'serializers': ['msgpack'],
                    'url':         'ws://localhost:8902/ws',
                    'max_retries': 3
                }
            ]
        )

    run([backend])

Client.py

import asyncio
from autobahn.asyncio.wamp import ApplicationSession
from autobahn.asyncio.component import Component, run


class Client(ApplicationSession):
    def __init__(self, config=None):
        config.realm = 'TESTS'
        super().__init__(config)


    async def onJoin(self, details):
        print('Call test_echo_payload')
        loop = asyncio.get_running_loop()
        loop.create_task(self.test_echo_payload())

    async def onDisconnect(self):
        loop  = asyncio.get_event_loop()
        loop.stop()

    async def test_echo_payload(self):
        message_bytes = b'\x01\x02\x03\x04\x05\x06\x07\x08\x09\x10'
        ret = await self.call('test.echo.payload', message_bytes)


if __name__ == '__main__':
    client = Component(
            session_factory = Client,
            transports      = [
                {
                    'type':        'websocket',
                    'serializers': ['msgpack'],
                    'url':         'ws://localhost:8902/ws',
                    'max_retries': 3
                }
            ]
        )

    run([client])

[Lois31]

Hello Tobias,

In the end everything works very well … I had to mix in my many tests … (Python, C++, Json, MsgPack, …)

Thank you again for your help. You can close this ticket.

[Lois31]

Thank you Tobias for your analysis and your reactivity!

In order to complete the COTS used, I tested with the V20.4.2 Crossbar WAMP router. Below, its configuration:

{
  "$schema": "https://raw.githubusercontent.com/crossbario/crossbar/master/crossbar.json",
  "version": 2,
  "controller": {
  },
  "workers": [
    {
      "type": "router",
      "realms": [
        {
          "name": "TESTS",
          "roles": [
            {
              "name": "anonymous",
              "permissions": [
                {
                  "uri": "",
                  "match": "prefix",
                  "allow": {
                    "call": true,
                    "register": true,
                    "publish": true,
                    "subscribe": true
                  },
                  "disclose": {
                    "caller": false,
                    "publisher": false
                  },
                  "cache": true
                }
              ]
            }
          ]
        }
      ],
      "transports": [
        {
          "type": "web",
          "endpoint": {
            "type": "tcp",
            "port": 8902,
            "backlog": 1024
          },
          "paths": {
            "/": {
              "type": "static",
              "directory": "../web",
              "options": {
                "enable_directory_listing": true
              }
            },
            "ws": {
              "type": "websocket",
              "serializers": [
                "msgpack"
              ],
              "options": {
                "allowed_origins": ["*"],
                "allow_null_origin": true,
                "enable_webstatus": false,
                "max_frame_size": 1048576,
                "max_message_size": 1048576,
                "auto_fragment_size": 65536,
                "fail_by_drop": true,
                "open_handshake_timeout": 2500,
                "close_handshake_timeout": 1000,
                "auto_ping_interval": 10000,
                "auto_ping_timeout": 5000,
                "auto_ping_size": 4,
                "compression": {
                  "deflate": {
                    "request_no_context_takeover": false,
                    "request_max_window_bits": 13,
                    "no_context_takeover": false,
                    "max_window_bits": 13,
                    "memory_level": 5
                  }
                }
              }
            },
            "info": {
              "type": "nodeinfo"
            }
          }
        }
      ]
    }
  ]
}

Let’s get rid of C++ and go back to my original problem. The problem being that when we send a message of type “bytes” while serializing with msgpack, we receive a message of type “str” …

I just did some additional tests which make me think that the problem comes from the router, I explain. Here are the steps of my test:

1. Start the Crossbar router on Linux1 machine
2. Star the Backend component on Linux2 machine
3. Start the Client component on Linux1 machine

The Client component correctly sends the byte table type message encoded in msgpack as follows: 0x9530ce0001e24080b1746573742e6563686f2e7061796c6f616491c40a01020304050607080910 But on the Backend component, we receive the message of the following form:

0000   18 60 24 80 ee d2 48 0f cf 35 d8 ac 08 00 45 00   .`$...H..5....E.
0010   00 76 8a 82 40 00 40 06 8c 66 2c 21 66 b9 2c 21   .v..@.@..f,!f.,!
0020   64 9e 22 c6 c0 e8 76 98 54 58 13 f5 16 52 80 18   d."...v.TX...R..
0030   00 85 b0 f0 00 00 01 01 08 0a 10 8f 32 63 7a ec   ............2cz.
0040   34 1c 82 40 95 44 19 12 81 a9 70 72 6f 63 65 64   4..@.D....proced
0050   75 72 65 b1 74 65 73 74 2e 65 63 68 6f 2e 70 61   ure.test.echo.pa
0060   79 6c 6f 61 64 91 bd 5b 22 5c 75 30 30 30 30 62   yload..["\u0000b
0070   27 41 51 49 44 42 41 55 47 42 77 67 4a 45 41 3d   'AQIDBAUGBwgJEA=
0080   3d 27 22 5d                                       ='"]

It seems that the Crossbar router returns the message in Json format and not Msgpack…