低精度的概念和意义 TensorFlow的FP16模型 TensorRT的FP16/Int8模型 总结 1 低精度的概念和意义 实数的16-bit半精度浮点数和8-bit定点数表示 使用低精度的意义 深度学习模型中实数的表示 FP32: E8M23 FP16: E8M7 FP16: E5M10 Int8 (TPU, tf.bfloat16) (tf.float32) (GPU, tf.float16) 低精度浮点数的优点 1.节约内存/显存的使用（FP16为原来的1/2，int8为原来的1/4） 2.特殊的硬件专门用于低精度浮点数的计算加速（TensorCore） Model Speedup BERT Q&A 3.3X speedup GNMT 1.7X speedup NCF 2.6X speedup ResNet-50-v1.5 3.3X speedup FP16浮点数（E5M10）的表示范围 FP16模型的训练方法 Int8模型的推断过程 2 TensorFlow的FP16模型 实数的16-bit半精度浮点数和8-bit定点数表示 使用低精度的意义 TensorCores适用条件 1. 卷积：K（输入通道），C（输出通道） 2. 通用矩阵乘法（GEMM）：MxK，KxN，（M，N，K） FP16: 大小为8x Int8: 大小为16x 如果FP32要使用，可以设置（内部转为FP16）：

foreign types: • Char, Int, Double, Float, and Bool as exported by the Haskell Prelude as well as • Int8, Int16, Int32, Int64, Word8, Word16, Word32, Word64, Ptr a, FunPtr a, and StablePtr a, for any type Constraint on concrete C type HsChar Char integral type HsInt Int signed integral type, ≥ 30 bit HsInt8 Int8 signed integral type, 8 bit; int8_t if available HsInt16 Int16 signed integral type, 16 bit; int16_t Char HS_INT_MIN minBound :: Int HS_INT_MAX maxBound :: Int HS_INT8_MIN minBound :: Int8 HS_INT8_MAX maxBound :: Int8 HS_INT16_MIN minBound :: Int16 HS_INT16_MAX maxBound :: Int16 HS_INT32_MIN minBound

primitive numeric types: • Integer types: Type Signed? Number of bits Smallest value Largest value Int8 ✓ 8 -2^7 2^7 - 1 UInt8 8 0 2^8 - 1 Int16 ✓ 16 -2^15 2^15 - 1 UInt16 16 0 2^16 - 1 Int32 ✓ 32 -2^31 FLOATING-POINT NUMBERS 16 julia> for T in [Int8,Int16,Int32,Int64,Int128,UInt8,UInt16,UInt32,UInt64,UInt128] println("$(lpad(T,7)): [$(typemin(T)),$(typemax(T))]") end Int8: [-128,127] Int16: [-32768,32767] julia> Int8(127) 127 julia> Int8(128) ERROR: InexactError: trunc(Int8, 128) Stacktrace: [...]CHAPTER 5. MATHEMATICAL OPERATIONS AND ELEMENTARY FUNCTIONS 35 julia> Int8(127.0) 127 julia> Int8(3.14)

primitive numeric types: • Integer types: Type Signed? Number of bits Smallest value Largest value Int8 ✓ 8 -2^7 2^7 - 1 UInt8 8 0 2^8 - 1 Int16 ✓ 16 -2^15 2^15 - 1 UInt16 16 0 2^16 - 1 Int32 ✓ 32 -2^31 FLOATING-POINT NUMBERS 16 julia> for T in [Int8,Int16,Int32,Int64,Int128,UInt8,UInt16,UInt32,UInt64,UInt128] println("$(lpad(T,7)): [$(typemin(T)),$(typemax(T))]") end Int8: [-128,127] Int16: [-32768,32767] julia> Int8(127) 127 julia> Int8(128) ERROR: InexactError: trunc(Int8, 128) Stacktrace: [...]CHAPTER 5. MATHEMATICAL OPERATIONS AND ELEMENTARY FUNCTIONS 35 julia> Int8(127.0) 127 julia> Int8(3.14)

primitive numeric types: • Integer types: Type Signed? Number of bits Smallest value Largest value Int8 ✓ 8 -2^7 2^7 - 1 UInt8 8 0 2^8 - 1 Int16 ✓ 16 -2^15 2^15 - 1 UInt16 16 0 2^16 - 1 Int32 ✓ 32 -2^31 (-2147483648, 2147483647) julia> for T in [Int8,Int16,Int32,Int64,Int128,UInt8,UInt16,UInt32,UInt64,UInt128] println("$(lpad(T,7)): [$(typemin(T)),$(typemax(T))]") end Int8: [-128,127] Int16: [-32768,32767] different forms. julia> Int8(127) 127 julia> Int8(128) ERROR: InexactError: trunc(Int8, 128) Stacktrace: [...] julia> Int8(127.0) 127 julia> Int8(3.14) ERROR: InexactError: Int8(3.14) Stacktrace:

primitive numeric types: • Integer types: Type Signed? Number of bits Smallest value Largest value Int8 ✓ 8 -2^7 2^7 - 1 UInt8 8 0 2^8 - 1 Int16 ✓ 16 -2^15 2^15 - 1 UInt16 16 0 2^16 - 1 Int32 ✓ 32 -2^31 (-2147483648, 2147483647) julia> for T in [Int8,Int16,Int32,Int64,Int128,UInt8,UInt16,UInt32,UInt64,UInt128] println("$(lpad(T,7)): [$(typemin(T)),$(typemax(T))]") end Int8: [-128,127] Int16: [-32768,32767] different forms. julia> Int8(127) 127 julia> Int8(128) ERROR: InexactError: trunc(Int8, 128) Stacktrace: [...] julia> Int8(127.0) 127 julia> Int8(3.14) ERROR: InexactError: Int8(3.14) Stacktrace:

primitive numeric types: • Integer types: Type Signed? Number of bits Smallest value Largest value Int8 ✓ 8 -2^7 2^7 - 1 UInt8 8 0 2^8 - 1 Int16 ✓ 16 -2^15 2^15 - 1 UInt16 16 0 2^16 - 1 Int32 ✓ 32 -2^31 (-2147483648, 2147483647) julia> for T in [Int8,Int16,Int32,Int64,Int128,UInt8,UInt16,UInt32,UInt64,UInt128] println("$(lpad(T,7)): [$(typemin(T)),$(typemax(T))]") end Int8: [-128,127] Int16: [-32768,32767] different forms. julia> Int8(127) 127 julia> Int8(128) ERROR: InexactError: trunc(Int8, 128) Stacktrace: [...] julia> Int8(127.0) 127 julia> Int8(3.14) ERROR: InexactError: Int8(3.14) Stacktrace:

primitive numeric types: • Integer types: Type Signed? Number of bits Smallest value Largest value Int8 ✓ 8 -2^7 2^7 - 1 UInt8 8 0 2^8 - 1 Int16 ✓ 16 -2^15 2^15 - 1 UInt16 16 0 2^16 - 1 Int32 ✓ 32 -2^31 (-2147483648, 2147483647) julia> for T in [Int8,Int16,Int32,Int64,Int128,UInt8,UInt16,UInt32,UInt64,UInt128] println("$(lpad(T,7)): [$(typemin(T)),$(typemax(T))]") end Int8: [-128,127] Int16: [-32768,32767] different forms. julia> Int8(127) 127 julia> Int8(128) ERROR: InexactError: trunc(Int8, 128) Stacktrace: [...] julia> Int8(127.0) 127 julia> Int8(3.14) ERROR: InexactError: Int8(3.14) Stacktrace:

primitive numeric types: • Integer types: Type Signed? Number of bits Smallest value Largest value Int8 ✓ 8 -2^7 2^7 - 1 UInt8 8 0 2^8 - 1 Int16 ✓ 16 -2^15 2^15 - 1 UInt16 16 0 2^16 - 1 Int32 ✓ 32 -2^31 (-2147483648, 2147483647) julia> for T in [Int8,Int16,Int32,Int64,Int128,UInt8,UInt16,UInt32,UInt64,UInt128] println("$(lpad(T,7)): [$(typemin(T)),$(typemax(T))]") end Int8: [-128,127] Int16: [-32768,32767] different forms. julia> Int8(127) 127 julia> Int8(128) ERROR: InexactError: trunc(Int8, 128) Stacktrace: [...] julia> Int8(127.0) 127 julia> Int8(3.14) ERROR: InexactError: Int8(3.14) Stacktrace:

primitive numeric types: • Integer types: Type Signed? Number of bits Smallest value Largest value Int8 ✓ 8 -2^7 2^7 - 1 UInt8 8 0 2^8 - 1 Int16 ✓ 16 -2^15 2^15 - 1 UInt16 16 0 2^16 - 1 Int32 ✓ 32 -2^31 (-2147483648, 2147483647) julia> for T in [Int8,Int16,Int32,Int64,Int128,UInt8,UInt16,UInt32,UInt64,UInt128] println("$(lpad(T,7)): [$(typemin(T)),$(typemax(T))]") end Int8: [-128,127] Int16: [-32768,32767] different forms. julia> Int8(127) 127 julia> Int8(128) ERROR: InexactError: trunc(Int8, 128) Stacktrace: [...] julia> Int8(127.0) 127 julia> Int8(3.14) ERROR: InexactError: Int8(3.14) Stacktrace: