Trouble saving CSR matrix to S3 using scipy.sparse.save_npz

I’m running into an issue where I tried to save a CSR matrix to aws s3 using the testing codes below:

import numpy as np
import pandas as pd
from scipy import sparse
import s3fs

df = pd.DataFrame(np.random.randint(0,100,size=(100, 4)), columns=list('ABCD'))
csr = sparse.csr_matrix(df.values)

s3 = s3fs.S3FileSystem(anon=False)
s3_path = "<an_aws_s3_path>"
f = s3.open(s3_path, 'wb')
sparse.save_npz(f, csr)

If this is a not current support function, could you provide any leads on good ways to achieve my goal?

Thanks.

Error trace:

...
  File "/Users/yangzhou/code/sml/core/util/csr_matrix_wrapper.py", line 37, in save
    sparse.save_npz(f, self.csr)
  File "/usr/local/lib/python3.7/site-packages/scipy/sparse/_matrix_io.py", line 78, in save_npz
    np.savez_compressed(file, **arrays_dict)
  File "/usr/local/lib/python3.7/site-packages/numpy/lib/npyio.py", line 667, in savez_compressed
    _savez(file, args, kwds, True)
  File "/usr/local/lib/python3.7/site-packages/numpy/lib/npyio.py", line 695, in _savez
    zipf = zipfile_factory(file, mode="w", compression=compression)
  File "/usr/local/lib/python3.7/site-packages/numpy/lib/npyio.py", line 112, in zipfile_factory
    return zipfile.ZipFile(file, *args, **kwargs)
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/zipfile.py", line 1214, in __init__
    self.fp.seek(self.start_dir)
  File "/usr/local/lib/python3.7/site-packages/s3fs/core.py", line 1278, in seek
    raise ValueError('Seek only available in read mode')
ValueError: Seek only available in read mode

[martindurant]

I suppose you’d prefer

    if not self.readable():
        raise OSError('Seek only available in read mode')

and then you don’t have to edit zipfile

[xuchen]

there you go, PR incoming https://github.com/intake/filesystem_spec/pull/238