Lecture Notes on Gaussian Discriminant Analysis, Naive Bayes and EM Algorithm Feng Li fli@sdu.edu.cn Shandong University, China 1 Bayes’ Theorem and Inference Bayes’ theorem is stated mathematically characterize the relationship through parameters θ = {P(X = x | Y = y), P(Y = y)}x,y. 2 Gaussian Discriminant Analysis In Gaussian Discriminate Analysis (GDA) model, we have the following as- sumptions: • A1: | Y = 0 ∼ N(µ0, Σ): The conditional probability of continuous random variable X given Y = 0 is a Gaussian distribution parameterized by µ0 and Σ, such that the corresponding probability density function

Lecture 5: Gaussian Discriminant Analysis, Naive Bayes and EM Algorithm Feng Li Shandong University fli@sdu.edu.cn September 27, 2023 Feng Li (SDU) GDA, NB and EM September 27, 2023 1 / 122 Outline Outline 1 Probability Theory Review 2 A Warm-Up Case 3 Gaussian Discriminate Analysis 4 Naive Bayes 5 Expectation-Maximization (EM) Algorithm Feng Li (SDU) GDA, NB and EM September 27, 2023 2 / 122 2023 37 / 122 Gaussian Distribution Gaussian Distribution (Normal Distribution) p(x; µ, σ) = 1 (2πσ2)1/2 exp � − 1 2σ2 (x − µ)2 � where µ is the mean and σ2 is the variance Gaussian distributions

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 643 17.3.2 Gaussian Blur . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 645 17.3.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 652 17.3.6 Selective Gaussian Blur . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 653 17.3.7 Circular Motion all-or-nothing selection. Note For technically oriented readers: feathering works by applying a Gaussian blur to the selection channel, with the specified blurring radius. 7.1.2 Making a Selection Partially

. . . . . . . . . 414 15.2.3 Gaussian Blur . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415 15.2.4 Selective Gaussian Blur . . . . . . . . . . . . Manipulation Program 77 / 653 Note For technically oriented readers: feathering works by applying a Gaussian blur to the selection channel, with the specified blurring radius. 7.1.2 Making a Selection Partially Duplicate the layer (producing a new layer above it). 2. Desaturate the new layer. 3. Apply a Gaussian blur to the result, with a large radius (100 or more). 4. Set Mode in the Layers dialog to Divide

y(i)), y(i) = θTx(i) + ϵ(i) The noise ϵ(i) is drawn from a Gaussian distribution ϵ(i) ∼ N(0, σ2) Each y(i) is drawn from the following Gaussian y(i)|x(i); θ ∼ N(θTx(i), σ2) The log-likelihood ℓ(θ) = and Bayesian Statistics September 20, 2023 17 / 25 Linear Regression: MAP Solution θ follows a Gaussian distribution θ ∼ N(0, λ2I) p(θ) = 1 (2πλ2)n/2 exp � −θTθ 2λ2 � and thus log p(θ) = n log 1 regularizer in MAP estimation For MAP, different prior distributions lead to different regularizers Gaussian prior on θ regularizes the ℓ2 norm of θ Laplace prior exp(C∥θ∥1) on θ regularizes the the ℓ1 norm

selection--by choosing Select Sharpen . For technically oriented readers: feathering works by applying a Gaussian blur to the selection channel, with the specified blurring radius. Making a selection partially image: Duplicate the layer (producing a new layer above it). Desaturate the new layer. Apply a Gaussian blur to the result, with a large radius (100 or more). Set Mode in the Layers dialog to Divide the magnitude or type of blurring. The most broadly useful of these is the Gaussian blur. (Don't let the word "Gaussian" throw you: this filter makes an image blurry in the most basic way.) It has

..................... 401 Antialias 402 Blur 402 Gaussian Blur 403 Motion Blur 404 Pixelize 406 Selective Gaussian Blur 406 Tileable Blur 407 Variable Blur 407 their kind donation of high-quality images for this book project) • Thom van Os (Images in Selective Gaussian Blur) • Eric Galluzzo and Christopher Macgowan (Proofreading) • Nicholas Lamb (Tip about selections) shape of the letters. This was done by copying the text layer, filling it with gray and applying Gaussian blur (with Keep Transparent unchecked). Organic Patterns The leaf pattern was created in Render/IfsCompose

lady’s layer, and then creating a clone layer. We then right-click and add a filter mask and use Gaussian blur set to 10 or so pixels. The clone layer is then put behind the original layer, and set to the horizontal and vertical', 'emboss horizontal only', 'emboss laplascian', 'emboss vertical only', 'gaussian blur', 'gaussiannoisereducer', 'gradientmap', 'halftone', 'height to normal', 'hsvadjustment', transitions by using intermediate color values If you want even smoother effects, well, just use blur. Gaussian blur to be exact. And there you go. That last little trick concludes this tutorial. Curve Brush

horizontal and vertical', 'emboss horizontal only', 'emboss laplascian', 'emboss vertical only', 'gaussian blur', 'gaussiannoisereducer', 'gradientmap', 'halftone', 'height to normal', 'hsvadjustment', transitions by using intermediate color values If you want even smoother effects, well, just use blur. Gaussian blur to be exact. And there you go. That last little trick concludes this tutorial. Curve Brush circumference at a distance of 100 pixels, while being 10 times smaller in length. Gaussian: distributes the particles using a gaussian or normal distribution [https://en.wikipedia.org/wiki/Normal_distribution]

assume " denote the noise and is independently and identically distributed (i.i.d.) according to a Gaussian distribution N(0, �2). The density of "(i) is given by f(✏) = 1 p 2⇡� exp ✓ � ✏2 2�2 ◆ Hence the least square in the linear model comes from the fact that the training data are sampled with Gaussian noise. 6