Pandas - Read_csv Scientific Notation Large Number

I am trying to read a csv file with pandas that has some rows in scientific notation. When it reads the values it is not capturing the true underlying number. When I re-purpose the

Solution 1:

Can't seem to reproduce your problem, but maybe this will work?

df = pd.read_csv('0_IDI_Submitter_out.csv', dtype={'INPUT: Extra 1':np.object_})

Also, check the dtypes of your dataframe:

result= df.dtypes
print(result)

Solution 2:

The problem appears to be that opening a CSV file in Excel which contains large numbers or strings that appear as large numbers like product codes, SKU's, UPC's etc are automatically converted into scientific notation. Once this has been done, you'll have to manually go into Excel and re-format but trying to do this from Pandas does not appear possible and the data integrity is lost.

However, if I never open the file in Excel and work on it purely through Pandas then it is all good. Similarly, if you work purely in Excel you're also good.

My ultimate conclusion is that when working with large numbers or strings that appear as large numbers like product codes or UPC's it is best not to mix pandas with Excel. As an alternative, I just started saving all my dataframes as pickle files instead of csv.

Hope that helps anyone in the future.

Thanks