Specify Correct Dtypes To Pandas.read_csv For Datetimes And Booleans

I am loading a csv file into a Pandas DataFrame. For each column, how do I specify what type of data it contains using the dtype argument? I can do it with numeric data (code at b

Solution 1:

There are a lot of options for read_csv which will handle all the cases you mentioned. You might want to try dtype={'A': datetime.datetime}, but often you won't need dtypes as pandas can infer the types.

For dates, then you need to specify the parse_date options:

parse_dates : boolean, list of ints or names, list of lists, or dict
keep_date_col : boolean, defaultFalse
date_parser : function

In general for converting boolean values you will need to specify:

true_values  : list  Values to consider asTrue
false_values : list  Values to consider asFalse

Which will transform any value in the list to the boolean true/false. For more general conversions you will most likely need

converters : dict. optional Dict of functions for converting values in certain columns. Keys can either be integers or column labels

Though dense, check here for the full list: http://pandas.pydata.org/pandas-docs/stable/generated/pandas.io.parsers.read_csv.html