#3: What is a throw expression in assembly? 0000800c : 800c: b508 push {r3, lr} 800e: 2004 movs r0, #4 8010: f000 f93c bl 828c <__cxa_allocate_exception> back into the registers. R5 R6 R7 R8 R9 R10 R11 R12 Scratch Register R13 / SP Stack Pointer R14 / LR Link Register (Updated with the return address of a subroutine call) Pushed on the stack if the function push {r4, lr} ; save R4 & LR ; ; Do stuff... ; Call a function at some point... ; 372c: 4620 mov r0, r4 ; Set return value ; Restore R4 back to R4 & LR into PC return

更新。该函数接受模型参数集合、学习速率和 批量大小作为输入。每一步更新的大小由学习速率lr决定。因为我们计算的损失是一个批量样本的总和，所 以我们用批量大小（batch_size）来规范化步长，这样步长大小就不会取决于我们对批量大小的选择。 98 3. 线性神经网络 def sgd(params, lr, batch_size): #@save """小批量随机梯度下降""" with with torch.no_grad(): for param in params: param -= lr * param.grad / batch_size param.grad.zero_() 3.2.7 训练 现在我们已经准备好了模型训练所有需要的要素，可以实现主要的训练过程部分了。理解这段代码至关重要， 因为从事深度学习后，相同的训练过程几乎一遍又一遍地出现。在每次迭代中，我们读取一小批量训练样本， 数据集，并将训练数据集中所有样本都使用 一次（假设样本数能够被批量大小整除）。这里的迭代周期个数num_epochs和学习率lr都是超参数，分别设 为3和0.03。设置超参数很棘手，需要通过反复试验进行调整。我们现在忽略这些细节，以后会在 11节中详细 介绍。 lr = 0.03 num_epochs = 3 net = linreg loss = squared_loss for

and Daniel Saks 4  Non-Member Function: _Z10terminatorP2HRP6SalaryP6HourlyP4Temp: PUSH {R4‐R6,LR} MOV R4,R0 MOV R5,R2 MOV R6,R3 BL _Z9terminateP2HRP6Salary MOV R1,R5 MOV R1,R6 MOV R0,R4 POP {R4‐R6,LR} B _Z9terminateP2HRP4Temp  Member Function: _ZN2HR10terminatorEP6SalaryP6HourlyP4Temp: PUSH {R4‐R6,LR} MOV R4,R0 MOV R5,R2 MOV MOV R0,R4 BL _ZN2HR9terminateEP6Hourly MOV R1,R6 MOV R0,R4 POP {R4‐R6,LR} B _ZN2HR9terminateEP4Temp 12 Identical Results, Up To Renaming Copyright © 2020 by Stephen

14.2 (-m32) x86 msvc v19.40 VS17.10 mov r0, #60 bx lr mov eax, 60 ret mov eax, 60 ret 0 i n t mov r1, #60 str r1, [r0] bx lr mov eax, DWORD PTR [esp+4] mov DWORD PTR [eax], 60 14.2 (-m32) x86 msvc v19.40 VS17.10 mov r0, #60 bx lr mov eax, 60 ret mov eax, 60 ret 0 i n t mov r1, #60 str r1, [r0] bx lr mov eax, DWORD PTR [esp+4] mov DWORD PTR [eax], 60 x86-64 gcc 14.2 (-m32) x86 msvc v19.40 VS17.10 mov r0, #60 bx lr mov eax, 60 ret mov eax, 60 ret 0 i n t mov r0, #60 bx lr mov eax, DWORD PTR [esp+4] mov DWORD PTR [eax], 60 ret 4

?0为初始学习率） 14 Pytorch的优化器 # 超参数 LR = 0.01 opt_SGD = torch.optim.SGD(net_SGD.parameters(), lr=LR) opt_Momentum = torch.optim.SGD(net_Momentum.parameters(), lr=LR, momentum=0.9) opt_RMSProp = torch torch.optim.RMSprop(net_RMSProp.parameters(), lr=LR, alpha=0.9) opt_Adam = torch.optim.Adam(net_Adam.parameters(), lr=LR, betas=(0.9, 0.99)) 15 Pytorch的优化器 16 神经网络的局部最优问题 17 局部最优问题 18 01 小批量梯度下降

6XicbVBNS8N AEJ3Ur1q/qh69LBbBU0lEUG9FLx4rGltoQ9lsN+3SzSb sToQS+hO8eFDx6j/y5r9x2+agrQ8GHu/NMDMvTKUw6Lr fTmldW19o7xZ2dre2d2r7h8miTjPskYluh9RwKRT 3UaDk7VRzGoeSt8LRzdRvPXFtRKIecJzyIKYDJSLBKFr pPu15vWrNrbs 6XicbVBNS8N AEJ3Ur1q/qh69LBbBU0lEUG9FLx4rGltoQ9lsN+3SzSb sToQS+hO8eFDx6j/y5r9x2+agrQ8GHu/NMDMvTKUw6Lr fTmldW19o7xZ2dre2d2r7h8miTjPskYluh9RwKRT 3UaDk7VRzGoeSt8LRzdRvPXFtRKIecJzyIKYDJSLBKFr pPu15vWrNrbs 6XicbVBNS8N AEJ3Ur1q/qh69LBbBU0lEUG9FLx4rGltoQ9lsN+3SzSb sToQS+hO8eFDx6j/y5r9x2+agrQ8GHu/NMDMvTKUw6Lr fTmldW19o7xZ2dre2d2r7h8miTjPskYluh9RwKRT 3UaDk7VRzGoeSt8LRzdRvPXFtRKIecJzyIKYDJSLBKFr pPu15vWrNrbs

的易扩展性）。 model.compile(loss=keras.losses.categorical_crossentropy, optimizer=keras.optimizers.SGD(lr=0.01, momentum=0.9, nesterov=True)) 现在，你可以批量地在训练数据上进行迭代了： # x_train 和 y_train 是 Numpy 数组 -- 就像在 Scikit-Learn activation='relu')) model.add(Dropout(0.5)) model.add(Dense(10, activation='softmax')) sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True) model.compile(loss='categorical_crossentropy', optimizer=sgd activation='relu')) model.add(Dropout(0.5)) model.add(Dense(10, activation='softmax')) sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True) model.compile(loss='categorical_crossentropy', optimizer=sgd)

Generators Create a parser from a grammar. Popular parsing algorithms used in generators are LL, LL(k), LR, LR(k), LALR, GLR... EBNF grammar example: identifier = alphabetic character, { alphabetic character Compile Time Parser Generator • Created by Piotr Winter • Written in C++17 • Library for generating LR(1) parsers from a grammar • Can generate a lexer or use custom one More information available at CppCast * 2 - 10")).value();LR(1) Parser Generator Overview Grammar Item Sets (states) Parsing tables Image from: “Compilers: Principles, Techniques, and Tools” (dragon book)CTPG – LR(1) Item struct item