Type annotations for DataLoader
See original GitHub issueI just started experimenting with aiodataloader
and it’s been great so far. Having type annotations (at least for the public DataLoader
class) would make it even easier for folks who are leveraging Python’s typing system.
From a preliminary look it seems like adding the basics for the DataLoader
would be pretty straightforward. We would need to accept a type for the KeyType
and ValueType
. I think it would look something like
from aiodataloader import DataLoader
class UserLoader(DataLoader[int, User]):
async def batch_load_fn(self, keys: Sequence[int]) -> Sequence[User]:
return await my_batch_get_users(keys)
user_loader = UserLoader()
I may have some time to try doing this myself. If I do would you be interested in accepting the patch?
Issue Analytics
- State:
- Created 2 years ago
- Reactions:7
- Comments:9 (2 by maintainers)
Top Results From Across the Web
Incorrect type annotation for DataLoader · Issue #52806 - GitHub
I believe the _collate_fn_t needs to be Optional[Callable[[List[T]], Any]]] . As it stands, it should not pass python type-checking (it fails ...
Read more >typing — Support for type hints — Python 3.11.1 documentation
Source code: Lib/typing.py This module provides runtime support for type hints. ... Introducing syntax for annotating variables outside of function ...
Read more >Writing data loaders - sfaira - Read the Docs
In brief, a data loader is a set of instructions that allows for streamlining of raw count matrices and meta data into objects...
Read more >Some Thoughts About Using Type Hints With PyTorch
The main disadvantage is that using type hints clutters up the code, making it more difficult to read.
Read more >Datasets & DataLoaders — PyTorch Tutorials 1.13.1+cu117 ...
PyTorch provides two data primitives: torch.utils.data.DataLoader and torch.utils.data.Dataset that allow you to use pre-loaded datasets as well ...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
Following up here, @aawilson has a PR that is nearing completion, and will result in a fully-typed release of aiodataloader (I expect this to be 0.3.0).
@erikwrede, ideally graphene should switch back to using this library directly, since it is in fact maintained and we are moving forward now with improvements. What do you think?
@allancaffee, I’ve already done work to implement typing on top of aiodataloader. How about I get a PR done in the next week or two, and you can take a look?