AttributeError: 'Blob' object has no attribute 'download_as_bytes'

Problem description

Be sure your description clearly answers the following questions:

• What are you trying to achieve? Trying to pull a file from gcp storage

• What is the expected result? I can read a file from gcp storage

• What are you seeing instead? Error: AttributeError: ‘Blob’ object has no attribute ‘download_as_bytes’

Steps/code to reproduce the problem

Failed on this line: for line in open(file, 'r'): I think it gets the file open and begins reading for the loop when it fails.

In order for us to be able to solve your problem, we have to be able to reproduce it on our end. Without reproducing the problem, it is unlikely that we’ll be able to help you.

Include full tracebacks, logs and datasets if necessary. Please keep the examples minimal (minimal reproducible example).

Apr 05 16:19:30 <host> docker[2234166]: File “/usr/local/lib/python3.7/site-packages/smart_open/gcs.py”, line 325, in read1 Apr 05 16:19:30 <host> docker[2234166]: return self.read(size=size) Apr 05 16:19:30 <host> docker[2234166]: File “/usr/local/lib/python3.7/site-packages/smart_open/gcs.py”, line 320, in read Apr 05 16:19:30 <host> docker[2234166]: self._fill_buffer(size) Apr 05 16:19:30 <host> docker[2234166]: File “/usr/local/lib/python3.7/site-packages/smart_open/gcs.py”, line 374, in _fill_buffer Apr 05 16:19:30 <host> docker[2234166]: bytes_read = self._current_part.fill(self._raw_reader) Apr 05 16:19:30 <host> docker[2234166]: File “/usr/local/lib/python3.7/site-packages/smart_open/bytebuffer.py”, line 152, in fill Apr 05 16:19:30 <host> docker[2234166]: new_bytes = source.read(size) Apr 05 16:19:30 <host> docker[2234166]: File “/usr/local/lib/python3.7/site-packages/smart_open/gcs.py”, line 178, in read Apr 05 16:19:30 <host> docker[2234166]: binary = self._download_blob_chunk(size) Apr 05 16:19:30 <host> docker[2234166]: File “/usr/local/lib/python3.7/site-packages/smart_open/gcs.py”, line 194, in _download_blob_chunk Apr 05 16:19:30 i<host> docker[2234166]: binary = self._blob.download_as_bytes(start=start, end=end) Apr 05 16:19:30 <host> docker[2234166]: AttributeError: ‘Blob’ object has no attribute ‘download_as_bytes’ Apr 05 16:19:30 <host> docker[2234166]: I0405 16:19:30.721222 139676694796096 gcs.py:241] close: called

Versions

Please provide the output of:

import platform, sys, smart_open
print(platform.platform())
print("Python", sys.version)
print("smart_open", smart_open.__version__)

Python 3.7.10 (default, Apr 2 2021, 22:24:51) [GCC 8.3.0] on linux Type “help”, “copyright”, “credits” or “license” for more information.

import platform, sys, smart_open print(platform.platform()) Linux-5.4.89±x86_64-with-debian-10.9 print(“Python”, sys.version) Python 3.7.10 (default, Apr 2 2021, 22:24:51) [GCC 8.3.0] print(“smart_open”, smart_open.version) smart_open 5.0.0

Tried on the latest 4 version too, same thing.

Checklist

Before you create the issue, please make sure you have:

• Described the problem clearly
• Provided a minimal reproducible example, including any required data
• Provided the version numbers of the relevant software

[gelioz]

Had the same issue. Pinning to the latest version of google-cloud-storage fixed the problem.

It would be nice to pin google-cloud-storage>=1.31.0 in smart_open[gcs] dependencies, as there was no Blob.download_as_bytes method before this release: https://github.com/googleapis/python-storage/releases/tag/v1.31.0

[petedannemann]

Ok I still can’t reproduce this at all and still have no idea what your file actually is. From this log Apr 05 16:19:30 docker[2234166]: I0405 16:19:30.721222 139676694796096 gcs.py:241] close: called it looks like somehow you already closed the Blob prior to reading from it?

Can you try a minimally reproducible example like this? This is a publically available file in GCS.

path = "gs://tensorflow-nightly/prod/tensorflow/release/ubuntu_16/gpu_py37_full/nightly_release/18/20190813-010608/github/tensorflow/pip_pkg/tf_nightly_gpu-1.15.0.dev20190813-cp37-cp37m-linux_x86_64.whl"

import smart_open
import google.cloud.storage

client = google.cloud.storage.Client.create_anonymous_client()
with smart_open.open(path, transport_params=dict(client=client)) as f:
    for line in f:
        print(line)