but as a reachable threshold. If / when achieved, AGI would redefine what software (and related hardware) can do. Rather than executing pre-programmed tasks, AGI systems would understand goals, generate storage, but for real-time inference and model training workloads that require dense, high-power hardware. As AI moves from experimental to essential, so too do data centers. Per NVIDIA Co-Founder and the same time, the cost of applying/using these models – known as inference – is falling quickly. Hardware is improving – for example, NVIDIA’s 2024 Blackwell GPU consumes 105,000x less energy per token

ChenCurrent Deep Learning Landscape Frameworks and Inference engines DL Compilers Kenrel Libraries Hardware CuDNN NNPack MKL-DNN Hand optimized Open source, automated end-to- end optimization framework Optimization AutoTVM Device FleetExisting Deep Learning Frameworks High-level data flow graph Hardware Primitive Tensor operators such as Conv2D eg. cuDNN Offload to heavily optimized DNN operator graph and optimizations Directly generate optimized program for new operator workloads and hardware Hardware FrameworksWhy Automation is the Future Clear winner on emerging models in product Competitive

INTERNATIONAL INDUSTRIES, INCORPORATED 8 Hardware Configure Chisel VTA for DE10-Nano DEPLOY VTA ON INTEL FPGA©2019 HARMAN INTERNATIONAL INDUSTRIES, INCORPORATED 9 Hardware Datapath of Chisel VTA DEPLOY VTA VTA ON INTEL FPGA©2019 HARMAN INTERNATIONAL INDUSTRIES, INCORPORATED 10 Hardware DEPLOY VTA ON INTEL FPGA©2019 HARMAN INTERNATIONAL INDUSTRIES, INCORPORATED 11 Getting Started DEPLOY VTA ON INTEL FPGA vta/config/de10nano_config.json to vta_config.json Step 9: Go to vta/hardware/intel and run make command Step 10: Get the generated .sof file programmed into hardware Step 11: Evaluate the unit test script test_vta_insn

been integrated by: ○ MATLAB Coder ○ ONNX RuntimeArm platform support in TVM upstream IPs Target Hardware/Model Options Codegen CPU arm_cpu pixel2 (snapdragon 835), mate10/mate10pro (kirin 970), p20/p20pro runtime plugins? ○ Integrate TVM codegen into Arm NN? ● CI and benchmark testing for TVM on member hardware platforms ○ Shall we maintain a list of Arm platforms supported by TVM? More details from our

relay.vm.compile Relay Object (hardware independent) Code segment VM Func 0 VM Func 1 ... VM Func N Data segment Const 0 Const 1 ... Const K Kernel lib (hardware dependent) Packed Func 0 Packed

our network, but also get a good performance gain by autotuning 3. TVM can support many kinds of hardware platform: Intel/arm CPU, Nividia/arm GPU, VTA…5 �������������� 1. Get a .log file from the autotvm

@ Intel GPU /NiiOS ! 驱动万物智能 8000% 7000% 6000% 5000% 4000% 3000% 2000% 1000% 0o0% GEMM Hardware Efficiency @ Intel Apollo Lake GPU 60.39% 512,512,512 国OpenVINO 国TVM 68.89% 1024 1024, 1024