Lecture 4: Regularization and Bayesian Statistics Feng Li Shandong University fli@sdu.edu.cn September 20, 2023 Feng Li (SDU) Regularization and Bayesian Statistics September 20, 2023 1 / 25 Lecture Regularization and Bayesian Statistics 1 Overfitting Problem 2 Regularized Linear Regression 3 Regularized Logistic Regression 4 MLE and MAP Feng Li (SDU) Regularization and Bayesian Statistics September 20, 2023 θ1x y = θ0 + θ1x + θ2x2 y = θ0 + θ1x + · · · + θ5x5 Feng Li (SDU) Regularization and Bayesian Statistics September 20, 2023 3 / 25 Overfitting Problem (Contd.) Underfitting, or high bias, is when the

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 6.4 Descriptive statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 6.5 Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 8.2 Moving (rolling) statistics / moments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 8.3 Exponentially (1-dimensional) and DataFrame (2-dimensional), handle the vast majority of typical use cases in finance, statistics, social science, and many areas of engineering. For R users, DataFrame provides everything that

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 6.4 Descriptive statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 6.5 Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 8.2 Moving (rolling) statistics / moments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 8.3 Exponentially (1-dimensional) and DataFrame (2-dimensional), handle the vast majority of typical use cases in finance, statistics, social science, and many areas of engineering. For R users, DataFrame provides everything that

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 6.4 Descriptive statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 6.5 Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 8.2 Moving (rolling) statistics / moments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 8.3 Exponentially (1-dimensional) and DataFrame (2-dimensional), handle the vast majority of typical use cases in finance, statistics, social science, and many areas of engineering. For R users, DataFrame provides everything that

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 8.5 Descriptive statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 8.6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211 10.2 Moving (rolling) statistics / moments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215 10.3 Expanding (1-dimensional) and DataFrame (2-dimensional), handle the vast majority of typical use cases in finance, statistics, social science, and many areas of engineering. For R users, DataFrame provides everything that

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203 9.5 Descriptive statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209 9.6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315 11.2 Moving (rolling) statistics / moments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319 11.3 Expanding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 622 25 Pandas Ecosystem 623 25.1 Statistics and Machine Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273 9.5 Descriptive statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279 9.6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 399 14.2 Moving (rolling) statistics / moments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403 14.3 Expanding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 772 29 pandas Ecosystem 773 29.1 Statistics and Machine Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265 9.5 Descriptive statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271 9.6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 389 14.2 Moving (rolling) statistics / moments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393 14.3 Expanding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 758 29 pandas Ecosystem 759 29.1 Statistics and Machine Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175 9.5 Descriptive statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182 9.6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285 11.2 Moving (rolling) statistics / moments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289 11.3 Expanding (1-dimensional) and DataFrame (2-dimensional), handle the vast majority of typical use cases in finance, statistics, social science, and many areas of engineering. For R users, DataFrame provides everything that

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337 10.5 Descriptive statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344 10.6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 479 15.2 Moving (rolling) statistics / moments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 483 15.3 Expanding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 895 30 pandas Ecosystem 897 30.1 Statistics and Machine Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .