How To Store `pandas.dataframe` In A Pandas-loadable Binary Format Other Than `pickle`

I have a problem with saving pandas.DataFrame (1 440 000 000 rows). From what I can see in the API, the only available options to store (and then load) the array are either CSV or

Solution 1:

I would guess that your data frame is too big. Pickle has some limits. You are much better off either saving in a database or using to_hdf (or lots of other IO routines, to_msgpack might works as well).

https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.to_hdf.html