. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3007 4.7.2 Checking memory leaks with valgrind . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3008 4.8 Extending Cabin 204 non-null object 11 Embarked 889 non-null object dtypes: float64(2), int64(5), object(5) memory usage: 83.7+ KB The method info() provides technical information about a DataFrame, so let’s explain considerations Disk vs memory pandas operates exclusively in memory, where a SAS data set exists on disk. This means that the size of data able to be loaded in pandas is limited by your machine’s memory, but also

(FAQ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 978 2.26.1 DataFrame memory usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 978 2.26.2 Using if/truth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2839 4.9.2 Checking memory leaks with valgrind . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2840 4.10 Extending Cabin 204 non-null object 11 Embarked 889 non-null object dtypes: float64(2), int64(5), object(5) memory usage: 83.7+ KB The method info() provides technical information about a DataFrame, so let’s explain

(FAQ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 978 2.26.1 DataFrame memory usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 978 2.26.2 Using if/truth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2837 4.9.2 Checking memory leaks with valgrind . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2838 4.10 Extending Cabin 204 non-null object 11 Embarked 889 non-null object dtypes: float64(2), int64(5), object(5) memory usage: 83.7+ KB The method info() provides technical information about a DataFrame, so let’s explain

(FAQ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 970 2.26.1 DataFrame memory usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 970 2.26.2 Using if/truth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2744 4.9.2 Checking memory leaks with valgrind . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2745 4.10 Extending Cabin 204 non-null object 11 Embarked 889 non-null object dtypes: float64(2), int64(5), object(5) memory usage: 83.7+ KB The method info() provides technical information about a DataFrame, so let’s explain

(FAQ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 969 2.26.1 DataFrame memory usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 969 2.26.2 Using if/truth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2744 4.9.2 Checking memory leaks with valgrind . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2745 4.10 Extending Cabin 204 non-null object 11 Embarked 889 non-null object dtypes: float64(2), int64(5), object(5) memory usage: 83.7+ KB The method info() provides technical information about a DataFrame, so let’s explain

(FAQ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 930 2.26.1 DataFrame memory usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 930 2.26.2 Using if/truth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2665 4.9.2 Checking memory leaks with valgrind . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2666 4.10 Extending Cabin 204 non-null object 11 Embarked 889 non-null object dtypes: float64(2), int64(5), object(5) memory usage: 83.7+ KB The method info() provides technical information about a DataFrame, so let’s explain

text_col 3 non-null object float_col 3 non-null float64 dtypes: float64(1), int64(1), object(1) memory usage: 152.0+ bytes pandas 1.0.0 In [34]: df = pd.DataFrame({"int_col": [1, 2, 3], ....: "text_col": (continued from previous page) 2 float_col 3 non-null float64 dtypes: float64(1), int64(1), object(1) memory usage: 200.0+ bytes 1.5.5 pandas.array() inference changes pandas.array() now infers pandas’ new option of downcasting the newly (or already) numeric data to a smaller dtype, which can conserve memory: In [396]: m = ['1', 2, 3] In [397]: pd.to_numeric(m, downcast='integer') # smallest signed int

previously evaluated the supplied function consistently twice on the first group to infer if it is safe to use a fast code path. Particularly for functions with side effects, this was an undesired behavior SparseArray([0, 0, 1, 2])}) In [68]: df.dtypes Out[68]: A Sparse[int64, 0] Length: 1, dtype: object The memory usage of the two approaches is identical. See Migrating for more (GH19239). 1.3.2 msgpack format • RangeIndex now performs standard lookup without instantiating an actual hashtable, hence saving memory (GH16685) • Improved performance of read_csv() by faster tokenizing and faster parsing of small

previously evaluated the supplied function consistently twice on the first group to infer if it is safe to use a fast code path. Particularly for functions with side effects, this was an undesired behavior SparseArray([0, 0, 1, 2])}) In [68]: df.dtypes Out[68]: A Sparse[int64, 0] Length: 1, dtype: object The memory usage of the two approaches is identical. See Migrating for more (GH19239). 1.3.2 msgpack format • RangeIndex now performs standard lookup without instantiating an actual hashtable, hence saving memory (GH16685) • Improved performance of read_csv() by faster tokenizing and faster parsing of small

(FAQ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 852 2.21.1 DataFrame memory usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 852 2.21.2 Using if/truth Cabin 204 non-null object 11 Embarked 889 non-null object dtypes: float64(2), int64(5), object(5) memory usage: 83.7+ KB The method info() provides technical information about a DataFrame, so let’s explain option of downcasting the newly (or already) numeric data to a smaller dtype, which can conserve memory: In [396]: m = ['1', 2, 3] (continues on next page) 1.4. Community tutorials 143 pandas: powerful