Not work with unicode as input

Example code as below:

def MyCompleter(prefix, **kwargs):
    results = ['aaa', 'bbb']
    return (c for c in results if c.startswith(prefix))

if __name__ == '__main__':
    import argparse
    import argcomplete

    parser = argparse.ArgumentParser()
    subparsers = parser.add_subparsers(dest='command')
    create_parser = subparsers.add_parser('new')
    create_parser.add_argument('title')
    create_parser.add_argument('category').completer = MyCompleter
    argcomplete.autocomplete(parser)
    args = parser.parse_args()
    print args

It works as expect when all the inputs are normal characters (assume above code are wrapped in command demo):

$ ./demo.py new 'hi'
--help  -h      aaa     bbb  

When the input contains unicode, the auto completer is not working:

$ ./demo.py new '你好'

So, is this a bug or something related to the shell? I’m using zsh and argcomplete (Version: 1.9.2).

[evanunderscore]

It looks like zsh is being smarter than bash and giving COMP_POINT in unicode chars instead of bytes. When we incorrectly interpret that number, we end up slicing the input halfway through a character, which we then fail to decode. Even if that didn’t crash, the results would likely be undesirable.

I used bashcompinit as described here to reproduce this. I’m assuming bashcompinit is supposed to emulate bash completion, in which case I’d say this should be considered a bug on their end.

On the other hand, the documentation for COMP_POINT explicitly says “If the current cursor position is at the end of the current command, the value of this variable is equal to ${#COMP_LINE}.” So bashcompinit is adhering to the spec and bash is not.

[evanunderscore]

The fix is now in zsh’s master branch (https://github.com/zsh-users/zsh/commit/e2f793e7df7214cc3d80e2fcfe961ed087c860ab).