non-dates are often converted to datetime
See original GitHub issueDescribed
issue where non-dates are often converted to datetime.
Environment OS: Windows 10 Python: 3.8.2 IDE: Scite
Package versions
pandasgui==0.2.11
pandas==1.1.4
I changed utility.py line to
` return pd.to_datetime(s, format="%b %d %Y")`
and now I only see
_PandasGUI WARNING — pandasgui.utility — In metadata_df, converted columns to datetime: RecordingDate_
which is desired.
It does convert everything else to a string (like float columns), but that is not a big issue for me, visually.
Here are the first two lines and header which are a result of `df.to_csv('metadata_df.csv')` which I do just before `show(metadata_df)`
[metadata_df.txt](https://github.com/adamerose/PandasGUI/files/6625776/metadata_df.txt)
Firstly, when I upgraded to 0.2.11 it seems to need a new module
`>pip install pandasgui==0.2.11`
I got `ModuleNotFoundError: No module named 'qtstylish'`
python Python 3.8.2 (tags/v3.8.2:7b3ab59, Feb 25 2020, 23:03:10) [MSC v.1916 64 bit (AMD64)] on win32
import pandas pandas.version ‘1.1.4’ import pandasgui Traceback (most recent call last): File “<stdin>”, line 1, in <module> File “C:\Python38\lib\site-packages\pandasgui_init_.py”, line 15, in <module> from pandasgui.gui import show File “C:\Python38\lib\site-packages\pandasgui\gui.py”, line 20, in <module> import qtstylish ModuleNotFoundError: No module named ‘qtstylish’ import pandasgui pandasgui.version ‘0.2.11’
but it did install fine: `pip install qtstylish`, so now I have
python Python 3.8.2 (tags/v3.8.2:7b3ab59, Feb 25 2020, 23:03:10) [MSC v.1916 64 bit (AMD64)] on win32
import pandas pandas.version ‘1.1.4’ import pandasgui pandasgui.version ‘0.2.11’
I did see the simple call in utility.py parse_dates_series(), so it appears to me now it is a pandas "thing", but maybe the question was why is pandasgui trying to recast columns at all?
So, it is likely that reading from CSV is not a proper test here.
The data was strings read from report forms, and the df types as created are
… Side object Location object NIHSScore float64 TICIScore object IV_tPA_timing object EVT_timing object RecordingDate object RecordingStartTime object RecordingStopTime object Gender object Age object SystolicBP object DiastolicBP object
my calling code saves that df to csv, then opens pandasgui on the dataframe for a look
metadata_df.to_csv('metadata_df.csv')
try:
show(metadata_df)
except ValueError:
pass
When I call show() I get:
![a](https://user-images.githubusercontent.com/20799053/121825595-3737c900-cc68-11eb-9ac0-550e9e204e55.jpg)
I might need to generate a pickle of metadata_df for you to test, if you'd like; if I "filter" the df through CSV, I get a result very similar to your screen cap, but with floats instead of ints
![b](https://user-images.githubusercontent.com/20799053/121825591-31da7e80-cc68-11eb-9d1f-594aed5f982c.jpg)
Small data pkl in a zip:
[head.zip](https://github.com/adamerose/PandasGUI/files/6644870/head.zip)
Issue Analytics
- State:
- Created 2 years ago
- Comments:5 (2 by maintainers)
Top GitHub Comments
Very nice, thanks!
Ah yeah you’re right, I convert columns to date if they’re
object
type because that’s often the type of columns containing strings, and I believe there’s no way to differentiate between typical string columns and other types ofobject
column like dict or whatnot, so I just try converting everyobject
column to a date.StringDtype
is actually unaccounted for, I’ll fix that.I loaded your pickle and ran
parse_dates
and these are the columns that got turned into dates on my end.So here are the issues I see:
parse_dates
is too aggressive and sometimes converts things to datetime when not desired. SOLUTION - I added an if-statement to check if the whole column is NaN and skip it in that caseStringDtype
doesn’t get parsed into a date SOLUTION - Check for this as well asobject