Ellipsis as dictionary literal key causes Syntax Error

Probably too specific to receive attention but maybe it highlights some internal design flaw or something:

Both in IPython console and Jupyter notebook cells (with ipython kernel), the following happens:

• opening brace in same line as ellipsis works fine

In [1]: { ...: 42 }
Out[1]: {Ellipsis: 42}

• ellipsis in beginning of line, without space before colon doesn’t get recognized (note the set({42}) returned)

In [2]: {
   ...: ...: 42
   ...: }
Out[2]: {42}

• ellipsis in beginning of line, with space before colon causes SyntaxError

In [3]: {
   ...: ... : 42
   ...: }
  File "<ipython-input-3-eb9728c6c108>", line 2
    : 42
    ^
SyntaxError: invalid syntax

• ellipsis after spaces, with space before colon works fine

In [4]: {
   ...:     ... : 42
   ...: }
Out[4]: {Ellipsis: 42}

• ellipsis after spaces, without space before colon doesn’t get recognized

In [5]: {
   ...:     ...: 42
   ...: }
Out[5]: {42}

• with any items before, it always work fine

In [6]: {
   ...:     'foo': 'bar', ...: 42
   ...: }
Out[6]: {'foo': 'bar', Ellipsis: 42}

• but items aftwards doesn’t make a difference on previous cases

In [7]: {
   ...:     ...: 42, 'foo': 'bar'
   ...: }
  File "<ipython-input-7-449ccd03925d>", line 2
    42, 'foo': 'bar'
             ^
SyntaxError: invalid syntax

And all these cases works fine in a source file ot in normal python repl (tested 3.5, 3.6, 3.7 and IPython 6.5.0)

I realize it’s a very specific bug with an easy workaround, and it’s probably similar to #2605 and #4059 , but I was working in jupyter using a match(obj, cases) function (eg: match(foo(), {'bar' : 42}), using ... for the default case and this bug tripped me so hard before I started to understand, it was worthy of this issue

[Carreau]

I believe I see where this is coming from. We try to do some magic to let user past code that already have an IPython prompt in the front.

@takluyver did work on the tokinsation, so may have some more insight, and does have plan to refactor.

Thanks for the exaustive use case, that would be of great help when we work on this.

[betafcc]

btw @takluyver now I think It’s possible to keep the ‘raw console paste’ functionality and fix these cases, at least if there is a preprocess step done ‘by cell’ instead of ‘by line’ (which I guess there is right? How does autoawait work? Wraps each line in functions?).

Just made a cell magic as POC, that simply uses ast.parse to check for syntax error before checking for leading ‘>>>’ and ‘…’ to strip, probably there are lighter ways to check it, like what is done in doctest.DocTestFinder but I am still a total newbie in ipython and a bit in python in general so I can’t reach a PR-worthy solution

edit

Just noticed it actually works by cell already, so the POC is:

import ast
from IPython.core.inputtransformer2 import PromptStripper

prev = PromptStripper.__call__

def wrapper(self, lines):
    try:
        ast.parse(''.join(lines))
        return lines
    except SyntaxError:
        return prev(self, lines)

PromptStripper.__call__ = wrapper

seems to work: