[tile] DB libraries, and large-scale graph analytics. Open-source software projects resulting from his work include the Matrix Template Library, the Boost Graph Library and Open MPI. 。Phil Ratzloff *。Distinguished relationships between elements of a data set *，Without regard to what the data set actually is *。 Graph theoretical (abstract) results can be applied to many different practical (concrete) problems -theory cvMuNKCATIOWS 有二有 灿瑟必帮 和一是/第 手太则光大 A，训四 昱 站 -本| 厅 是，主 半一司、|全之体 ~ 对 /人-看 奋才全 和 全原 和 刁曾 The Future ls Big Graph: on meetalThngs soneaiadman MD] Basic Principles *。The C++ standard library (nee STL) provides

Scott, Principal Engineer at CMU SEI Graph/ML/AI algorithms for large- and small- scale parallel systems. Working on GBTL, a linear algebra-based C++ library for graph analytics.[DISTRIBUTION STATEMENT GraphBLAS community, C API Overview of our draft C++ API How might this interoperate with standard C++, graph library proposal? 4[DISTRIBUTION STATEMENT A] This material has been approved for public release GraphBLAS community, C API Overview of our draft C++ API How might this interoperate with standard C++, graph library proposal? 5[DISTRIBUTION STATEMENT A] This material has been approved for public release

top-down task graph What is Task Graph-based Programming System (TGPS) Code 4• TGPS encapsulates function calls and their dependencies in a top-down task graph What is Task Graph-based Programming task graph What is Task Graph-based Programming System (TGPS) Code A B C D B A C D 6• TGPS encapsulates function calls and their dependencies in a top-down task graph What is Task Graph-based precede(task_d); 19 20 sched.schedule(); 21 sched.wait(); B A C D 1. Easy to write and express a task graph 2. Allow to implement irregular parallel decomposition strategies 8Existing TGPSs on Heterogenous

ingests a FP32 graph and a small dataset • Finds suitable quantization scale • Produces a quantized graph • Compiling Pre-quantized models – QNN Dialect • TVM ingests a pre-quantized graph in TFLite or rights reserved. TVM Overview Framework Graph Mxnet TF …. parsers Relay Graph Target-independent Relay passes Target-optimized graph Target-dependent Relay passes Intel x86 ARM CPU Nvidia GPU targets AutoTVM – Tuning the kernels Optimized Binary Codegen – LLVM, Cuda, C, … Framework Parsers Graph level optimizations Tensor-level optimizations Machine code generation© 2019, Amazon Web Services

from tvm import relay 2. Load a pretrained network mod, params = relay.testing.mobilenet.get_workload(batch_size=1) 3. Partition and build the network with an external codegen mod = relay.build_extern(mod build_extern(mod, “dnnl”) 4. Run the inference exe = relay.create_executor(“vm”, mod=mod, ctx=tvm.cpu(0)) data = np.random.uniform(size=(1, 3, 224, 224)).astype(“float32”) out = exe.evaluate()(data, **params) How System Overview Relay IR Graph Annotation with Your Annotator Graph Partitioning Your Codegen LLVM, CUDA, Metal, VTA Serialized Subgraph Library Relay Runtime (VM, Graph Runtime, Interpreter)

dependent: arange, nms, etc. ○ Control flow: concatenate within a while loop Limitation of TVM/graph runtime ● Cannot compile and run dynamic models© 2019, Amazon Web Services, Inc. or its Affiliates at runtime ● Virtual machine as a new runtime for Relay ● Dynamic codegen (WIP) ○ Kernel dispatch for a single op ○ Graph dispatch for a (sub-)graph In collaboration with Jared Roesch, Zhi Chen, Wei Wei Chen© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. “Any” in Relay typing Any: represent an unknown dimension at compilation time. Define a tensor type: Tensor<(Any, 3, 32

Tensor Graph Optimization Framework Tensor Graph to Xilinx Tensor Graph Frontend Deep Learning Frameworks https://github.com/xilinx© Copyright 2018 Xilinx TVM as Unified ML Front End >> 6 Relay (and (and NNVM) Graph Parser XIR Compiler Quantizer Partitioner @relay.transform.module_pass(opt_level=4) class AccelModule:© Copyright 2018 Xilinx TVM Partitioning >> 7 Subgraph 1 Parallel Subgraphs supported/not supported, pattern matching graph colorization - Choices how to partition especially for multi-branch networks (i.e. YOLOv3, SSD)© Copyright 2018 Xilinx TVM Graph Partitioning/Fusion >> 8 Subgraph

ASIC Optimization AutoTVM Device FleetExisting Deep Learning Frameworks High-level data flow graph Hardware Primitive Tensor operators such as Conv2D eg. cuDNN Offload to heavily optimized intensiveMachine Learning based Program Optimizer TVM: Learning-based Learning System High-level data flow graph and optimizations Directly generate optimized program for new operator workloads and hardware module/pass, type system, with function variants supportCompilation Flow under the New Infra IRModule (relay::Function) IRModule (te::Function, ExternFunc, …) runtime::Module High-level optimizations (Auto)

OpenAI- Add relay.nn.sparse_dense for block-sparse matrix multiplication (~50 lines of TVM IR) - Add relay.reinterpret to implement rational approximations in user space (~10 lines of Relay IR) - A few icache/ dcache - also available today in FBGEMMPyTorch and TVM - Lots of opportunity in PyTorch - Graph optimization - Existing fusion infrastructure fairly limited (CUDA-only, injective-only) - Kernel synthesis - Dynamic shapes, stride specialization - Impedance mismatch with PyTorch JIT IR and Relay IR - Watch this space :)Big thanks to the community

TVM @ Hexagon DSP 人NiOS ! 驱动万物知 Tensorflow deploy.so / deploy.json / deploy.bin | NNVM / Relay 让 Graph Optimization 站 站 Compile | libtvm_hexagon_runtime.so Alios TVM @ Hexagon DSP 。 Compute