Downloading https://files.pythonhosted.org/packages/a8/76/220ba4420459d9c4c9c9587c6ce607bf5 6c25b3d3d2de62056efe482dadc /seaborn-0.9.0-py3-none-any.whl (208kB) 100% |████████████████████████████████| Downloading https://files.pythonhosted.org/packages/c3/8b/af9e0984f5c0df06d3fab0bf396eb09cb f05f8452de4e9502b182f59c33b/ matplotlib-3.1.1-cp37-cp37m- macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64 using below formula and then find the weights using normal distribution, stddev = sqrt(scale / n) where n represent,  number of input units for mode = fan_in  number of out units for mode = fan_out

size. Two transformations applied separately result in a dataset 3x the original size. Can we apply N transformations to create a dataset Nx the size? What are the constraining factors? An image transformation forward model. Figure 3-12: An example of back-translation. EN=>DE model in this case is facebook/wmt19-en-de and DE=>EN model is facebook/wmt19-de-en. Table 3-4 shows the performance comparison of regular

(SDU) K-Means December 28, 2021 2 / 46 Clustering Usually an unsupervised learning problem Given: N unlabeled examples {x1, · · · , xN}; no. of desired partitions K Goal: Group the examples into K “homogeneous” Problem Given a set of observations X = {x1, x2, · · · , xN} (xi ∈ RD), partition the N observations into K sets (K ≤ N) {Ck}k=1,··· ,K such that the sets minimize the within-cluster sum of squares: arg over all points defines the K-means “loss function” L(µ, X, Z) = N � i=1 K � k=1 zi,k∥xi − µk∥2 = ∥X − Zµ∥2 where X is N × D, Z is N × K and µ is K × D Feng Li (SDU) K-Means December 28, 2021 19 /

can separate the positive class and the negative class using the find command: % find returns the i n d i c e s of the % rows meeting the s p e c i f i e d condition pos = find (y == 1 ) ; neg = find will have to define it yourself. The easiest way to do this is through an inline expression: g = i n l i n e ( ’ 1.0 ./ ( 1 . 0 + exp(−z ) ) ’ ) ; % Usage : To find the value of the sigmoid % evaluated threshold ϵ, i.e. |L+(θ) − L(θ)| ≤ ϵ (7) Try to resolve the logistic regression problem using gradient de- scent method with the initialization θ = 0, and answer the following questions: 1. Assume ϵ = 10−6

w iw N w 1x 2x ix N x . . . . . . f y ? = ? ෍ ?=1 ? ???? + ? 6 1.人工神经网络发展历史 1982年，加州理工学院J.J.Hopfield 教授提出了Hopfield神经网络模型 ，引入了计算能量概念，给出了网 络稳定性判断。 离散Hopfield神经网络模型 1T 2T IT N T … … ELM)，是由黄广斌提出的用于处理单隐层 神经网络的算法 优点： 1.学习精度有保证 2.学习速度快 随机初始化输入权重??和偏置 ，只求解输出权重值??。 1 nx 1 ? ? i  n 1  i L  1  L  ny 1个输出 层神经元 ?个隐藏 层神经元 ?个输入 层神经元 9 2.感知器算法 01 发展历史 02 感知机算法 03 线性分类模型。 用 ? ∈ ??×? 表示数据集，用 ? 表示标 签。 需要学习的目标函数是 从一堆输入输出中学习模型参数?和?。  1 w b 2 w iw N w 1x 2x ix N x . . . . . . f y 输入 权重 偏置 求和 求和 输出 ?(?) = sign(?T? + ?) 11 2.感知机算法 感知机算法（Perceptron

学习语言模型 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303 8.3.2 马尔可夫模型与n元语法 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305 8.3.3 自然语言统计 . . . . . . • I：单位矩阵 • xi, [x]i：向量x第i个元素 • xij, [X]ij：矩阵X第i行第j列的元素 集合论 • X: 集合 • Z: 整数集合 • R: 实数集合 • Rn: n维实数向量集合 • Ra×b: 包含a行和b列的实数矩阵集合 • A ∪ B: 集合A和B的并集 13 • A ∩ B：集合A和B的交集 • A \ B：集合A与集合B相减，B关于A的相对补集 列的准确性，因为模型在开始生成新序列之前不再 需要记住整个序列。 • 多阶段设计。例如，存储器网络 (Sukhbaatar et al., 2015) 和神经编程器‐解释器 (Reed and De Freitas, 2015)。它们允许统计建模者描述用于推理的迭代方法。这些工具允许重复修改深度神经网络的内部状 态，从而执行推理链中的后续步骤，类似于处理器如何修改用于计算的存储器。 • 另一个关键的发展是生成对抗网络

then p(y | x; θ) = 1 1 + exp(−yθTx) Assuming the training examples were generated independently, we de- fine the likelihood of the parameters as L(θ) = m � i=1 p(y(i) | x(i); θ) = m � i=1 (hθ(x(i)))y(i)(1 However, each iteration is more expensive than the one of gradient descent Finding and inverting an n × n Hessian More details about Newton’s method can be found at https://en.wikipedia.org/wiki/Newton%27s_method

Animation https://medium.freecodecamp.org/an-intuitive-guide-to-convolutional-neural- networks-260c2de0a050 Notation Input_channels: Kernel_channels: 2 ch Kernel_size: Stride: Padding: Multi-Kernels multi-k: [16, 3, 3, 3] bias: [16] out: [b, 16, 28, 28] LeNet-5 Pyramid Architecture http://cs231n.stanford.edu/slides/2016/winter1516_lecture7.pdf nn.Conv2d Inner weight & bias F.conv2d 下一课时 池化层

配套视频课程(收费，提供答疑等全服务，比较适合初学者)： 深度学习与 TensorFlow 入门实战 深度学习与 PyTorch 入门实战 https://study.163.com/course/courseMai n.htm?share=2&shareId=48000000184 7407&courseId=1209092816&_trace_c _p_k2_=9e74eb6f891d47cfaa6f00b5cb torch.randn([1, n]) cpu_b = torch.randn([n, 1]) print(n, cpu_a.device, cpu_b.device) # 创建使用 GPU 运算的 2 个矩阵 gpu_a = torch.randn([1, n]).cuda() gpu_b = torch.randn([n, 1]).cuda() cuda() print(n, gpu_a.device, gpu_b.device) 接下来实现 CPU 和 GPU 运算的函数，并通过 timeit.timeit()函数来测量两个函数的运 算时间。需要注意的是，第一次计算时一般需要完成额外的环境初始化工作，因此这段时 间不能计算在内。通过热身环节将这段时间去除，再测量运算时间，代码如下： def cpu_run(): # CPU

Receptive Field https://medium.freecodecamp.org/an-intuitive-guide-to-convolutional-neural- networks-260c2de0a050 Weight sharing ▪ ~60k parameters ▪ 6 Layers http://yann.lecun.com/exdb/publis/pdf/lecun-89e