UnicodeDecodeError when using sync_map.ldap and Python 2.7.13

The following ldap2pg.yml is working fine with Python 2.7.13 on Debian 9.4 (see attached ldap2pg.success.log for details):

verbose: on

ldap:
  uri: ldap://localhost:389/

postgres:
  managed_roles_query: |
    SELECT rolname
    FROM pg_catalog.pg_roles
    WHERE
        (rolsuper IS FALSE)
        AND (rolname NOT IN ('xxx-admin'))
    ORDER BY 1;

sync_map:
- roles:
  - name: role1
    options: NOLOGIN
  - name: role2
    option: NOLOGIN

But whenever I add ldap entries to the sync_map:

sync_map:
- roles:
  - name: role1
    options: NOLOGIN
  - name: role2
    option: NOLOGIN
- ldap:
    base: dc=cxptc,dc=org
  role:
    name_attribute: sAMAccountName
    options: LOGIN

I get this:

[...]
Querying LDAP dc=cxptc,dc=org... (objectClass...
Doing: ldapsearch -x -D  -b dc=cxptc,dc=org -s sub '(objectClass=*)' 
Unhandled error:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/ldap2pg/script.py", line 84, in main
    exit(wrapped_main(config))
  File "/usr/local/lib/python2.7/dist-packages/ldap2pg/script.py", line 68, in wrapped_main
    count = manager.sync(syncmap=config['sync_map'])
  File "/usr/local/lib/python2.7/dist-packages/ldap2pg/manager.py", line 188, in sync
    ldaproles, ldapacl = self.inspect_ldap(syncmap)
  File "/usr/local/lib/python2.7/dist-packages/ldap2pg/manager.py", line 127, in inspect_ldap
    entries = self.query_ldap(**mapping['ldap'])
  File "/usr/local/lib/python2.7/dist-packages/ldap2pg/manager.py", line 37, in query_ldap
    base, scope, filter, attributes,
  File "/usr/local/lib/python2.7/dist-packages/ldap2pg/ldap.py", line 161, in search_s
    return self.wrapped.search_s(base, scope, filter, attributes)
  File "/usr/local/lib/python2.7/dist-packages/ldap2pg/ldap.py", line 132, in __call__
    return decode_value(self.callable_(*a, **kw))
  File "/usr/local/lib/python2.7/dist-packages/ldap2pg/utils.py", line 86, in decode_value
    return [decode_value(v) for v in value]
  File "/usr/local/lib/python2.7/dist-packages/ldap2pg/utils.py", line 88, in decode_value
    return tuple([decode_value(v) for v in value])
  File "/usr/local/lib/python2.7/dist-packages/ldap2pg/utils.py", line 83, in decode_value
    for k, v in value.items()
  File "/usr/local/lib/python2.7/dist-packages/ldap2pg/utils.py", line 86, in decode_value
    return [decode_value(v) for v in value]
  File "/usr/local/lib/python2.7/dist-packages/ldap2pg/utils.py", line 79, in decode_value
    return value.decode('utf-8')
  File "/usr/lib/python2.7/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xbb in position 5: invalid start byte

The logs for successful and failed runs are attached: ldap2pg.fail.log ldap2pg.success.log

My config is: ldap2pg 4.10 from git master, psycopg2 2.6.2, python-ldap 2.4.28 and Python 2.7.13 on Debian 9.4. LDAP (Samba AD DC 4.8.2-debian) and PostgreSQL (9.6.7) are on the same host and anonymous LDAP searches are enabled.

I tried with Python 3.5.3 but encountered another issue: #224

[jrjsmrtn]

@bersace: I tested the git master with the sAMAccountName and userPrincipalName attributes: it’s working fine ! Thanks a lot for your help 😃

[bersace]

@jrjsmrtn nice. 4.11 is on PyPI with the fix ! Thanks for the report !

running upload
Submitting dist/ldap2pg-4.11.tar.gz to https://upload.pypi.org/legacy/
Server response (200): OK
Submitting /home/bersace/src/dalibo/ldap2pg/dist/ldap2pg-4.11-py2.py3-none-any.whl to https://upload.pypi.org/legacy/
Server response (200): OK