BUG: Small numbers in scientific notation do not show the right significant figures in DataFrame.

Research

• I have searched the [pandas] tag on StackOverflow for similar questions.

• I have asked my usage related question on StackOverflow.

Link to question on StackOverflow

https://stackoverflow.com/questions/71359050/pandas-series-not-showing-small-numbers-in-scientific-notation-depending-on-firs/71359216#71359216

Question about pandas

I am using pandas dataframes to store astronomical data (this is my library) and the default unit system has many variables in scientific notation. I would like to use the to_string() function to save the data as a text file. This is an example of the workflow:

import numpy as np
import pandas as pd

myDF = pd.DataFrame(columns=['c0', 'c1', 'c2', 'c3'])

myDF.loc['r0', 'c0'] = 'abc'
myDF.loc['r0', 'c1'] = np.nan
myDF.loc['r0', 'c2'] = 1234.0
myDF.loc['r0', 'c3'] = 1.234e-18

# Save the table as a dataframe.
with open(f'output_DF.txt', 'wb') as output_file:
    string_DF = myDF.to_string()
    output_file.write(string_DF.encode('UTF-8'))

print('Output DF\n', myDF)

Calling the cell data from the dataframe returns the right value (in this case 1.234e-18) but the dataframe is stored in a text file as:

      c0   c1     c2   c3
r0  abc  NaN  1234.0  0.0

This is also how it is displayed on the terminal. I think that the right behaviour should preserve the significant figures. I am not sure if this is an issue with the dtypes in the dataframe creation or while writting the output file.

I am using pandas 1.4.1 in windows 10. I wonder if anyone could please offer me some advice on the right way to do this operation.

Thank you for your work.

[mroeschke]

@ArLough I would recommend not tackling an issue with any of the Needs ... tags as further discussion might be needed to determine the action item. Instead I would suggest looking for issues with good first issue

[ArLough]

Ok, thank you! That’s helpful