Question: use cases (beyond which unit test lib I use)

Hi,

@mrbean-bremen answered a question of mine in StackOverflow, and I got interested in this lib. I am a long time python user, but unfortunately I did not give the required attention to the importance of testing. I know basics (difference of fake, stub, mock, what is a unit of code, etc).

I am practicing testing with a new project that uses pytest, and it has a module that reads XLS reports using pandas and make consolidations on numbers. I think it has potential usage to test many/most methods there, but I am not really sure since documentation says: pyfakefs will not work with Python libraries (other than os and io), but I suspect pandas uses os and io under the hood.

I also would like to know if you recommend any good materials on unit testing with fakes/pyfakefs/pytest specifically.

[mrbean-bremen]

@knightjdr , @staticdev: I just had another look at pandas, and at least for read_csv there is a possibility to use it with pyfakefs - just to let you know in case you are interested…

Pandas internally uses its TextFileReader to read from CSV, and this has an argument engine which defines which backend is used for file system access. The default is “c”, but if you set it to “python”, it will use the standard file system access and thus can be used with pyfakefs.

Patching TextFileReader accordingly will do the trick:

from unittest import mock

import pandas as pd
from pandas.io.parsers import TextFileReader

class PythonTextFileReader(TextFileReader):
    def __init__(self, *args, **kwargs):
        kwargs['engine'] = 'python'
        super().__init__(*args, **kwargs)

@mock.patch('pandas.io.parsers.TextFileReader', PythonTextFileReader)
def test_load_csv(fs):
    path = '/foo/bar.csv'
    fs.create_file(path, contents='1,2,3,4')
    df = pd.read_csv(path)
    assert (df.columns == ['1', '2', '3', '4']).all()

With pytest, this can be done using a separate fixture:

@pytest.fixture
def pd_fs(fs):
    with mock.patch('pandas.io.parsers.TextFileReader', PythonTextFileReader):
        yield fs

def test_load_csv(pd_fs):
    path = '/foo/bar.csv'
    fs.create_file(path, contents='1,2,3,4')
    df = pd.read_csv(path)
    assert (df.columns == ['1', '2', '3', '4']).all()

Unfortunately, this does not help for reading excel files - for that it uses the engine from one of several external packages. I will have a look at this later… would be nice to be able to do something here, as pandas is used a lot nowadays, and maybe integrate it into pyfakefs.

[staticdev]

@mrbean-bremen I will make tests ASAP. Unfortunately, too busy this week. It was added to my backlog =p