Affiliates. All rights reserved. Animesh Jain Amazon SageMaker Neo Compilation of Quantized Models in TVM AWS AI© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Quantization Overview Services, Inc. or its Affiliates. All rights reserved. Quantization in TVM • Quantization within TVM - Automatic Quantization • TVM stack ingests a FP32 graph and a small dataset • Finds suitable quantization QNN Dialect • TVM ingests a pre-quantized graph in TFLite or MxNet • Use high-level wrapper ops of QNN dialect© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. TVM Overview Framework