manager.py chokes on special characters in dn

If the distinguished name of a user contains special characters, e.g. CN=Markus Gräßer,OU=my_group,dc=example,dc=com (name changed), execution of ldap2pg fails in apply_role_rules. The following error is thrown:

[ldap2pg.script   CRITICAL] Failed to process ('CN=Markus Gr\xc3\xa4\xc3\x9fer: 'ascii' codec can't decode byte 0xc3 in position 12: ordinal not in range(128) 

I worked around it by changing apply_role_rules

def apply_role_rules(self, rules, entries):
    for rule in rules:
        for raw_entry in entries:
            entry = (
                raw_entry[0].decode('unicode_escape').encode('iso8859-1').decode('utf8'),
                raw_entry[1]
            )
            try:
                for role in self.process_ldap_entry(entry=entry, **rule):
                    yield role
            except ValueError as e:
                msg = "Failed to process %.32s: %s" % (entry, e,)
                raise UserError(msg)

The decoding line is a shameless copy from https://stackoverflow.com/a/6956914 (my python is limited).

This is on Centos 7.4.1708 with epel and pgdg-96-centos repository. ldap2pg was build according to https://ldap2pg.readthedocs.io/en/latest/install/ (the rpm in the pgdp-96 repo does not include all dependencies, but that is a bug report for the maintainer of the rpm).

[dirks]

@bersace, I changed a few values to make the entries anonymous in the example below. If you need an original entry I will send it via mail if that is fine with you. I did not notice earlier that the dn returned by ldapsearch is given as base64 string, if it contains Umlaute (ä, ß in this case).

---
color: yes
verbose: yes
dry: yes
 
ldap:
    uri: ldap://ad1.example.com ldap://ad2.example.com
    binddn: cn=svc.mybindusers,ou=Functional Users,ou=Users,ou=EXAMPLE,dc=example,dc=com
    password: "redacted"
 
postgres:
    dsn: ""
    blacklist: [postgres, pg_*, testing]

sync_map:
    __all__:
    - ldap:
        base: ou=AAI,dc=example,dc=com
        filter: "(&(objectclass=User)(sAMAccountname=abc12de))"
        attributes: sAMAccountName
      role:
        name_attribute: sAMAccountName
        options: LOGIN

# extended LDIF
#
# LDAPv3
# base <ou=AAI,dc=example,dc=com> with scope subtree
# filter: (&(objectclass=User)(sAMAccountname=abc12de))
# requesting: sAMAccountName 
#

# Markus Gr\C3\A4\C3\9Fer, Admins, Employees, AAI, example.com
dn:: Q049TWFya3VzIEdyw6TDn2VyLE9VPUFkbWlucyxPVT1FbXBsb3llZXMsT1U9QUFJLERDPWV4YW1wbGUsREM9Y29tCg==
sAMAccountName: ab12cde

# search result
search: 5
result: 0 Success

# numResponses: 2
# numEntries: 1

[dirks]

@bersace, yes the dry run works as expected. I continue deployment and testing this week. Thanks a lot. 😃