question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

why does smart_open.s3.Reader doesn't implement __enter__ and __exit__ method

See original GitHub issue

Problem description

Hi, I am trying to add alicloud oss support. While I am reading source code of smart_open.s3.Reader, I found smart_open.s3.Reader doesn’t implement __enter__ and __exit__ method.

How is it possible to write code like:

with open('s3://commoncrawl/robots.txt', 'rb') as fin:
    for line in fin:
        print(repr(line.decode('utf-8')))
        break

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:5 (2 by maintainers)

github_iconTop GitHub Comments

1reaction
mpenkovcommented, Oct 8, 2021

The reader/writer in smart_open.s3 does not need to implement the context manager protocol, because it gets wrapped by other classes that implement that protocol.

The example code you posted actually works. Have a look at the readme - it’s the main use case.

0reactions
njn007commented, Oct 9, 2021

@mpenkov , thanks for your explanation, I checked the python io docs, finding that S3.Reader -> io.BufferedIOBase -> io.IOBase. While IOBase is also a context manager and therefore supports the with statement which means that you don’t need to implement __enter__ method if you just want to return the reader itself. the __enter__ method is implemented in base class io.IOBase def __enter__(): return self

The reader/writer in smart_open.s3 does not need to implement the context manager protocol, because it gets wrapped by other classes that implement that protocol.

Read more comments on GitHub >

github_iconTop Results From Across the Web

smart_open/s3.py at develop · RaRe-Technologies ... - GitHub
"""Implements file-like objects for reading and writing from/to AWS S3.""" ... We use the above as a guide only, and do not perform...
Read more >
Source code for smart_open.smart_open_lib
The main methods are: * `smart_open()`, which opens the given file for reading/writing * `s3_iter_bucket()`, which goes over all keys in an S3...
Read more >
smart-open - PyPI
smart_open is a Python 3 library for efficient streaming of very large files from/to storages such as S3, GCS, Azure Blob Storage, HDFS,...
Read more >
Read and write from S3 server (ECS) using Python (Pyspark)
Read and write from S3 server (ECS) using Python (Pyspark) ... However when i do not user smart_open then it says path does...
Read more >
GuardDuty S3 finding types - AWS Documentation
If this activity is unexpected for the associated principal, it may indicate that the credentials have been exposed or your S3 permissions are...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found