Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Would it be worth making this method accept both filepath and BytesIO/StringIO object?

See original GitHub issue

When calling the to_parquet method in pandas, the api is not consistent when using fastparquet engine. For example, to_csv accepts either a filepath or a buffer. So does to_parquet when using the pyarrow engine. However, calling to_parquet when using the fastparquet engine will error when passed a buffer rather than filepath.

I think it would be good to adhere to the polymorphic api exposed by the other to_<format> methods and allow both filepath and buffer to be passed to to_parquet.

https://github.com/dask/fastparquet/blob/5b4f30a0a7fe90cc61ca5119b8a65ca614aa0a45/fastparquet/writer.py#L735

I am happy to submit a PR for this unless anyone has an objection?

Issue Analytics

State:
Created 5 years ago
Comments:8 (4 by maintainers)

Top GitHub Comments

1reaction

martindurantcommented, Oct 3, 2018

You should not need --doctest-cython, but install the source as you normally would do, e.g.,

pip install -e .

(run from the source directory).

0reactions

ubdussamadcommented, Sep 23, 2019

@martindurant Let me check, i have a clue to what maybe wrong, though I’ll learn conda and try to install it fresh through that, but if that still dosent works I’ll raise an issue.