Machine Learning Lecture 10: Neural Networks and Deep Learning Feng Li fli@sdu.edu.cn https://funglee.github.io School of Computer Science and Technology Shandong University Fall 2018 Deep Feedforward f(x) is usually a highly non-linear function • Feedforward networks are of extreme importance to machine learning practioners • The conventional neural networks (CNN) used for object recognition from photos

Machine Learning Pytorch Tutorial TA : 曾元（Yuan Tseng） 2022.02.18 Outline ● Background: Prerequisites & What is Pytorch? ● Training & Testing Neural Networks in Pytorch ● Dataset & Dataloader ● Tensors year ■ ref: link1, link2 Some knowledge of NumPy will also be useful! What is PyTorch? ● An machine learning framework in Python. ● Two main features: ○ N-dimensional Tensor computation (like NumPy) NLP & speech) ○ ESPnet (speech recognition, translation, synthesis, ...) ○ Most implementations of recent deep learning papers ○ ... References ● Machine Learning 2021 Spring Pytorch Tutorial ● Official

Lecture Notes on Support Vector Machine Feng Li fli@sdu.edu.cn Shandong University, China 1 Hyperplane and Margin In a n-dimensional space, a hyper plane is defined by ωT x + b = 0 (1) where ω ∈ Rn defined as γ = min i γ(i) (6) 1 ? ? ! ? ! Figure 1: Margin and hyperplane. 2 Support Vector Machine 2.1 Formulation The hyperplane actually serves as a decision boundary to differentiating positive we can construct a infinite number of hyperplanes, but which one is the best? Supported Vector Machine (SVM) answers the above question by maximizing γ (see Eq. (6)) as follows max γ,ω,b γ s.t. y(i)(ωT

Lecture 6: Support Vector Machine Feng Li Shandong University fli@sdu.edu.cn December 28, 2021 Feng Li (SDU) SVM December 28, 2021 1 / 82 Outline 1 SVM: A Primal Form 2 Convex Optimization Review (b < 0 means in opposite direction) Feng Li (SDU) SVM December 28, 2021 3 / 82 Support Vector Machine A hyperplane based linear classifier defined by ω and b Prediction rule: y = sign(ωTx + b) Given: " such that min& ' & !() & + " = 1 Feng Li (SDU) SVM December 28, 2021 14 / 82 Support Vector Machine (Primal Form) Maximizing 1/∥ω∥ is equivalent to minimizing ∥ω∥2 = ωTω min ω,b ωTω s.t. y(i)(ωTx(i)

similar to the model that is discussed in the Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation paper. This model script is available on GitHub and NGC. similar to the model that is discussed in the Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation paper. PyTorch Release 23.06 PyTorch RN-08516-001_v23.07 similar to the model that is discussed in the Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation paper. This model script is available on GitHub and NGC.

examples / more than two features? In those cases, we could use classical machine learning algorithms like the Support Vector Machine4 (SVM) to learn classifiers that would do this for us. We could rely on embeddings for the inputs using machine learning algorithms of your choice. 2. Embedding Lookup: Look up the embeddings for the inputs in the embedding table. 4 Support Vector Machine - https://en.wikipedia. org/wiki/Support-vector_machine 3. Train the model: Train the model for the task at hand5 with the embeddings as input. Refer to Figure 4-4 that describes the three steps visually. Figure 4-4: A high-level

start off on our journey to more efficient deep learning models. Introduction to Deep Learning Machine learning is being used in countless applications today. It is a natural fit in domains where there problems where we expect exact optimal answers, machine learning applications can often tolerate approximate responses, since often there are no exact answers. Machine learning algorithms help build models, which Relation between Artificial Intelligence, Machine Learning, and Deep Learning. Deep learning is one possible way of solving machine learning problems. Machine learning in turn is one approach towards

关系可能太复杂（比如像素和抽象类 别之间的关系），需要数千或数百万次的计算。即使人类的眼睛能毫不费力地完成这些难以提出完美解决方 案的任务，这其中的计算也超出了人类意识理解范畴。机器学习（machine learning，ML）是一类强大的可 以从经验中学习的技术。通常采用观测数据或与环境交互的形式，机器学习算法会积累更多的经验，其性能 17 也会逐步提高。相反，对于刚刚所说的电子商务平 1)是输入特征的一个 仿射变换（affine transformation）。仿射变换的特点是通过 加权和对特征进行线性变换（linear transformation），并通过偏置项来进行平移（translation）。 给定一个数据集，我们的目标是寻找模型的权重w和偏置b，使得根据模型做出的预测大体符合数据里的真实 价格。输出的预测值由输入特征通过线性模型的仿射变换决定，仿射变换由所选权重和偏置确定。 相匹配，我们将使用 缓存的文件，以避免重复的下载。 70 https://discuss.d2l.ai/t/1822 71 https://archive.ics.uci.edu/ml/machine‐learning‐databases/housing/housing.names 180 4. 多层感知机 def download(name, cache_dir=os.path.join('

quality is an important benchmark to evaluate the performance of a deep learning model. A language translation application that uses a low quality model would struggle with consumer adoption because it wouldn’t speak different languages. An application that employs a high quality model with a reasonable translation accuracy would garner better consumer support. In this chapter, our focus will be on the techniques techniques use models to generate samples for labels. Consider a training sample for English to Spanish translation: [English: “I am doing really well”, Spanish: “Estoy muy bien”]. Let’s say we have another model

the total frame number), and the tracking success ratio after initialization. Group A: simple translation Group B: there are loops Group C: slow and nearly pure rotation Group D: fast motion with strong Consistent Depth Maps Recovery from a Video Sequence. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 31(6):974-988, 2009. 典型应用 • 三维重建 • 视频场景编辑 三维重建 三维重建 视频场景编辑 软件或源代码 •