Support chunksize in I/O functionalities

Describe the problem

Chunksize defaults to pandas in all cases. Ideally, we will be able to support it. The functionality should not require deep analysis, we can just mask the result of reading with chunksize=None.

Relevant conversation in #909.

Source code / logs

Proposed solution for read_sql:

def read_sql(
    sql,
    con,
    index_col=None,
    coerce_float=True,
    params=None,
    parse_dates=None,
    columns=None,
    chunksize=None,
):
    _, _, _, kwargs = inspect.getargvalues(inspect.currentframe())
    kwargs["chunksize"] = None
    df = DataFrame(query_compiler=BaseFactory.read_sql(**kwargs))
    if chunksize is not None:
        num_chunks = len(df) // chunksize if len(df) % chunksize == 0 else len(df) // chunksize + 1
        return (df.iloc[i * chunksize: (i + 1) * chunksize] for i in num_chunks)
    else:
        return df

[eavidan]

PRs are always welcomed @manesioz 😃

I believe the yield-based approach fits the chunking principle of pandas better. For instance, querying in small chucks should result in a quick response from the database during the first iteration. It is expected that each iteration starts with a delay for getting its respectful chunk. Waiting a long time for the first iteration to start is not an expected behavior with a small chunksize. Moreover, the user may decide to stop iterating at some point, and this will not improve performance if everything was loaded into memory already.

[devin-petersohn]

Why don’t we try to implement with fetchmany. It may not work, but it will be worth trying. Otherwise, the right approach would be to use @eavidan’s approach and throw a warning about the order.