Disable output / silent option?

Would it be possible to have an option to disable any output / info messages when running the Profiler? I’ve had a look at the sources, and it seems simple prints are used, e.g. for:

Finding the Null values in the columns... (with 9 processes)
...

Maybe there could be a ‘silent=True’ option in the Profiler base class, or something similar? Having this output in the (standard) log(s) would still be useful, but as it is it gets in the way of my ‘normal’ command-line output, unless I apply some dodgy workarounds…

[makkus]

@ChrisWallace2020 thanks, much appreciated! I just tried out the latest version from git, and it works exactly as advertised for me, once I switchto logging.WARNING.

Actually, there was one warning still intact:

/home/markus/projects/dharpa/.direnv/python-3.9.6/lib/python3.9/site-packages/dataprofiler/profilers/profile_builder.py:584: RuntimeWarning: 

!!! WARNING Partial Profiler Failure !!!

Profiling Type: data_labeler
Exception: ModuleNotFoundError
Message: No module named 'tensorflow'

For labeler errors, try installing the extra ml requirements via:

$ pip install dataprofiler[ml] --user


  utils.warn_on_profile('data_labeler', e)

But I can turn that off easily by disabling the data_labeler in the profile options. It’s probably a good idea to do that explicitly anyway – even if though I deliberately did not install tensorflow, so I did not expect this to work. And since it’s a different sort of warning it probably makes sense to leave that one in.

Thanks again!

[ChrisWallace2020]

@makkus the 2 PRs above (#357, #368) should resolve the issue.

The prints were replaced with logging.info so that the effective level may be set high enough to filter them out if you choose to do so. This can be set either through logging like normal or you can use the function set_verbosity we’ve written in dp_logging.py that takes in the logging level (logging.DEBUG, logging.INFO, logging.WARNING etc). DataProfiler-specific logging will output to stdout (if left verbose).

I hope this addresses your issue, let me know if there is any further feedback. If this addresses your initial issue feel free to close this out, please and thank you.