Infer value type from parametrized (PEP-484) dict type?

Consider the following example code that sums the values of a dictionary that maps strings to floats:

def f():
    d: Dict[str, float]
    d = {}

    total: float
    total = 0.0

    # k: str  # I would like to avoid declaring these
    # v: float

    for k in d:
        v = d[k]  # Secondarily, I am also curious on how to accelerate this
        total += v

I find that if I manually declare v to be a float that Cython will nicely remove some Python object code. However, this might also be inferred from the declaration d: Dict[str, float]. I suspect that this kind of type inference is just out of scope for Cython, but I thought I’d ask to see if this was possible somehow.

Also, any additional tips for accelerating repeated dict access like this would be very welcome.

[scoder]

First of all, I agree that inferring the item type(s) of containers (especially when they are declared via PEP-484 typing) would be nice, and there’s probably already a hugely outdated ticket for that which was written years before PEP-484. Cython’s support for PEP-484 typing is definitely still young and in flux. It will have to mature over time. Feedback is always appreciated, as it allows us to see actual use cases that we can improve.

Regarding your example, inferring float would be helpful, since it can be safely aliased to C double. But that’s the special case. If the type was int, it wouldn’t help, since Cython cannot safely map Python ints to any fixed size C integer type. You could use cython.int or cython.long for that, though, which request specific C integer semantics.

Regarding str keys, there is no speed advantage to be gained for the loop you showed, and as Robert suggested, iterating only over d.values() generates much more efficient code already - if that suits your real use case. But also in other cases, str means “str or a subtype” in Python, but Cython can only make use of “exactly str” for optimisation, as subtypes can override the builtin type behaviour in arbitrary and unpredictable ways. If you type a variable as cdef str k in Cython, it will actually reject subtypes on assignment, for exactly that reason. That’s why we currently ignore PEP-484 style annotations of type str. They simply don’t have sufficiently tight semantics, similar to the int issue above.

The same subtyping issue applies also to a Dict annotation vs. cdef dict d, but this is again a special case that could be exploited at least for looping. The runtime special casing code is already mostly there as we apply an optimistic optimisation whenever we see code that loops over something.items() etc., which has a very high probability of something turning out to be exactly a dict in practice.

[da-woods]

@jakirkham That commit specifically accelerates the loop operation (by treating the type as a hint). It doesn’t apply the type anywhere else in Cython. (It’s also mentioned in one of the comments above)