Lecture Notes on Linear Regression Feng Li fli@sdu.edu.cn Shandong University, China 1 Linear Regression Problem In regression problem, we aim at predicting a continuous target value given an input assume a n-dimensional feature vector is denoted by x 2 Rn, while y 2 R is the output variable. In linear regression models, the hypothesis function is defined by h✓(x) = ✓nxn + ✓n�1xn�1 + · · · + ✓1x1 2 m X i=1 ⇣ h✓(x(i)) � y(i)⌘2 Our linear regression problem can be formulated as min ✓ J(✓) = 1 2 m X i=1 ⇣ ✓T x(i) � y(i)⌘2 1 Figure 1: 3D linear regression. Specifically, we aim at minimizing

Experiment 1: Linear Regression August 27, 2018 1 Description This first exercise will give you practice with linear regression. These exercises have been extensively tested with Matlab, but they should as an option in the installer, and available for Linux from Octave-Forge ). 2 Linear Regression Recall that the linear regression model is hθ(x) = θT x = n � j=0 θjxj, (1) where θ is the parameter extra intercept item x0 = 1. Therefore, the resulting feature vector is (n + 1)-dimensional. 1 3 2D Linear Regression We start a very simple case where n = 1. Download data1.zip, and extract the files (ex1x

Lecture 2: Linear Regression Feng Li Shandong University fli@sdu.edu.cn September 13, 2023 Feng Li (SDU) Linear Regression September 13, 2023 1 / 31 Lecture 2: Linear Regression 1 Supervised Learning: Classification 2 Linear Regression 3 Gradient Descent Algorithm 4 Stochastic Gradient Descent 5 Revisiting Least Square 6 A Probabilistic Interpretation to Linear Regression Feng Li (SDU) Linear Regression area (feet2) Price (1000$s) 2104 400 1600 330 2400 369 1416 232 3000 540 ... ... Feng Li (SDU) Linear Regression September 13, 2023 3 / 31 Supervised Learning (Contd.) Features: input variables, x;

• A short history – linear algebra and C++ (1998 – Present) • The Eigen C++ Template Library for Linear Algebra • Linear Algebra Interface in C++26 • Basics • Using with Eigen Outline Daniel Hanson Hanson CppCon 2024 2• This presentation is on solving problems using • The Eigen linear algebra library • stdBLAS in C++26 • Not affiliated with Eigen but have used it in financial programming and teaching 1998: • C++ growing in popularity • Wide adoption in financial programming • But, no support for linear algebra (pining for Fortran…) • Your options essentially were: • Write your own Matrix class and

12 std::linalg: Linear Algebra Coming to Standard C++ Mark Hoemmen, NVIDIA | CppCon 20233 Agenda • Motivating example: Matrix multiply • Where std::linalg fits in linear algebra’s layers • std::linalg matrix_product(par_unseq, scaled(alpha, A), transposed(B), scaled(beta, C), C);5 Does a “linear algebra library” do linear algebra? Aspirational linearity My impression of René Magritte’s “The Treachery of Rank-2 array (linear function between 2 vector spaces, assuming a basis for each) Products, norms, solves Computations with array inputs & {array or scalar} output Matrices are linear functions;

Better cache performance. ● Collisions are resolved via probing. Probing methods include: ○ Linear Probing. ○ Quadratic Probing. ○ Double Hashing. ○ Robin Hood Hashing.Open Addressing Hash Maps are resolved via probing. Probing methods include: ○ Linear Probing. ○ Quadratic Probing. ○ Double Hashing. ○ Robin Hood Hashing.Linear Probed Robin Hood Hashing 0 0 0 0 0 0 0 0 0 PSL element. 7. Repeat steps 5 and 6 until an empty bucket( bucket with default PSL value) is found.Linear Probed Robin Hood Hashing 0 0 0 0 0 0 0 0 0 PSL PSL - Probe Sequence Length X X X

. . . . . . . . . . 474 35.29 Multithreading and linear algebra . . . . . . . . . . . . . . . . . . . . . . . . . . . 475 35.30 Alternative linear algebra backends . . . . . . . . . . . . . . . . . Utilities 1369 75 Lazy Artifacts 1379 76 LibCURL 1380 77 LibGit2 1381 78 Dynamic Linker 1429 79 Linear Algebra 1433 79.1 Special matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1711 94.6 Sparse Linear Algebra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1711 95 SparseArrays API 1713 96 Sparse Linear Algebra API 1729 97 Noteworthy External

. . . . . . . . . . 474 35.29 Multithreading and linear algebra . . . . . . . . . . . . . . . . . . . . . . . . . . . 475 35.30 Alternative linear algebra backends . . . . . . . . . . . . . . . . . Utilities 1369 75 Lazy Artifacts 1379 76 LibCURL 1380 77 LibGit2 1381 78 Dynamic Linker 1429 79 Linear Algebra 1433 79.1 Special matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1711 94.6 Sparse Linear Algebra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1711 95 SparseArrays API 1713 96 Sparse Linear Algebra API 1729 97 Noteworthy External

. . . . . . . . . . 474 35.29 Multithreading and linear algebra . . . . . . . . . . . . . . . . . . . . . . . . . . . 475 35.30 Alternative linear algebra backends . . . . . . . . . . . . . . . . . Utilities 1369 75 Lazy Artifacts 1379 76 LibCURL 1380 77 LibGit2 1381 78 Dynamic Linker 1429 79 Linear Algebra 1433 79.1 Special matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1711 94.6 Sparse Linear Algebra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1711 95 SparseArrays API 1713 96 Sparse Linear Algebra API 1729 97 Noteworthy External

区块链网络上的任务执行节点： $ ./requester-cli nodes list 发布训练任务 训练任务由计算需求方发起： $ ./requester-cli task publish -a "linear-vl" -l "MEDV" --keyPath './keys' -t "train" -n "房价预测任务v3" -d "hahahha" -p "id,id" --conf ./tes executor2" # 命令行返回 TaskID: fdc5b7e1-fc87-4e4b-86ee-b139a7721391 命令行各参数说明如下： -a: 训练使用的算法, 可选线性回归 ‘linear-vl’ 或逻辑回归 ‘logistic-vl’ -l: 训练的目标特征 –keyPath: 默认取值’./keys’, 从该文件夹中读取私钥, 计算需求方的私钥, 表 明了计算需求方的身份, 可以用-k 使用的配置文件 发布预测任务 训练任务执行完成后产出预测模型，计算需求方可以提交预测任务，为预测数 据计算出预测结果。 $ ./requester-cli task publish -a "linear-vl" --keyPath './keys' -t "predict" -n "房价任务v3" -d "hahahha" -p "id,id" --conf ./testdata/exec