question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Problem with `.to_dict()` type inference

See original GitHub issue

Describe the bug

I have a dataframe or pandas series and when I use the .to_dict() the inferred type by pandas-stubs is Dict[Hashable, Any]. In my case I know that that the index is an int but there is an error when I type the output as Dict[int, Any] saying that int is not a Hashable type.

Is there any solution for better inferring the type in this case?

To Reproduce

Please complete the following information:

  1. Provide a minimal runnable pandas example that is not properly checked by the stubs.
import pandas as pd

id=[1,2,3]
value=["a", "b", "c"]

df = pd.DataFrame(zip(id, value), columns=["id", "value"])
df.set_index("id")["value"].to_dict()

def funct(df_dict: Dict[int, Any]) -> None:
     pass
  1. Indicate which type checker you are using (mypy or pyright).3. Mypy

  2. Show the error message received from that type checker while checking your example.4.

error: Argument 4 to "funct" has incompatible type "Dict[Hashable, Any]"; expected "Dict[int, Any]"

Additional context I am running it in python version 3.10

Issue Analytics

  • State:open
  • Created 9 months ago
  • Reactions:1
  • Comments:5 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
twoertweincommented, Dec 22, 2022

DataFrame/Series are not Generic with respect to the index type, so we cannot return dict[int, Any. I think we would need to relax the annotation to just dict to avoid any mypy/pyright errors when users implicitly “cast” it to dict[int, Any]. I’d be okay with that change (for input arguments, I would still require something like dict[HashableT, Any] but for return values, it might be worth relaxing it).

@Dr-Irv @bashtage

0reactions
twoertweincommented, Dec 24, 2022

Mapping makes only the values covariant. I would prioritize getting the container type right (dict vs (Mutable)Mapping) as opposed to differentiating Any vs Hashable.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Assignment to TypedDict field does not handle bidirectional ...
PEP 589 is very clear that a TypedDict class should not allow non-keyword arguments passed to its constructor, so pylance is correct in ......
Read more >
Generics: Why can't the compiler infer the type arguments in ...
I wanted to write an extension-method that would work on dictionaries whose values were some sort of sequence. Unfortunately, the compiler can't ...
Read more >
Lightwood API Types — lightwood 22.12.1.0 documentation
A populated ProblemDefinition object. to_dict(encode_json=False) ...
Read more >
(PDF) Type4Py: Deep Similarity Learning-Based Type ...
In this paper, we present Type4Py, a deep similarity learning-based type inference model for Python. We design a hierarchical neural network ...
Read more >
IO tools (text, CSV, HDF5, …) — pandas 1.5.2 documentation
Type inference is a pretty big deal. If a column can be coerced to integer dtype without altering the contents, the parser will...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found