Golang在BFE的应用 百度运维部 陶春华 taochunhua@baidu.com 2016年4月 个人简介 • 陶春华，运维部，Baidu Front End团队 –2010年，天津大学，计算机专业博士 • 2013年7月，加入百度 –使用GO开发的项目 • 7层流量代理GO-BFE • 应用层防火墙WAF • 百度GOLANG委员会成员 内容提要 • 后台程序开发的需求和难点 –严格的编译阶段检查：强类型，文件包含，… –Panic，便于定位问题 Golang(3) • 上线和运维 –可编译为独立可执行程序 BFE(Baidu Front End) • 百度统一前端 –七层流量接入平台 BFE(Baidu Front End) • 主要服务 • 接入转发 • 防攻击、流量调度、数据分析 • 业务现状 • 覆盖大部分重要产品 • 日请求量千亿级别 online query BFE GO-BFE 一个例子 • url encode case 1. http://xxx.baidu.com/item/JELIM+PLASTIC+SURGERY+%26+AESTHET IC 2. http://xxx.baidu.com/item/JELIM+PLASTIC+SURGERY+&+AESTHETIC 分布式架构 • BFE程序结构：core+众多功能模块

位操作函数 字符串函数 条件函数 数学函数 JSON解析函数 类型转换函数 格式转换函数 通用函数 时间和日期函数 BITMAP函数 HLL函数 窗口函数 哈希函数 Baidu 百度智能云文档 目录 2 SQL手册 数据类型 TINYINT数据类型 TINYINT数据类型 长度: 长度为1个字节的有符号整型。 范围: [-128, 127] 转换: Dor +------------------------+ +------------------------+ 1 row in set (0.00 sec) 1 row in set (0.00 sec) Baidu 百度智能云文档 SQL手册 3 BIGINT数据类型 BIGINT数据类型 长度: 长度为8个字节的有符号整型。 范围: [-9223372036854775808, 9223372036854775807] +------------------------------------------+ 1 row in set (0.05 sec) 1 row in set (0.05 sec) Baidu 百度智能云文档 SQL手册 4 转换: Doris不会自动将DOUBLE类型转换成其他类型。用户可以使用CAST()将其转换成TINYINT, SMALLINT, INT, BIGINT,

with Hub Service Message handling with Function Service Message Synchronize between baetyl-hub and Baidu IoTHub via Remote Service Image capturing and AI model inference with Video infer Service Development etc. The combination of Baetyl and the Cloud Management Suite of BIE [https://cloud.baidu.com/product/bie.html](Baidu IntelliEdge) will achieve cloud management and application distribution, enable applications many other productions of Baidu Cloud(such as CFC [https://cloud.baidu.com/product/cfc.html], Infinite [https://cloud.baidu.com/product/infinite.html], EasyEdge [https://ai.baidu.com/easyedge/home], TSDB

Local Function Service Workflow Message Handling Test Message Synchronize between Baetyl-Hub and Baidu IoTHub via Baetyl- Remote-MQTT module Workflow Message Synchronize via Baetyl-Remote-MQTT module etc. The combination of Baetyl and the Cloud Management Suite of BIE [https://cloud.baidu.com/product/bie.html](Baidu IntelliEdge) will achieve cloud management and application distribution, enable applications many other productions of Baidu Cloud(such as CFC [https://cloud.baidu.com/product/cfc.html], Infinite [https://cloud.baidu.com/product/infinite.html], EasyEdge [https://ai.baidu.com/easyedge/home], TSDB

. . . . . . . . . . . . . . . . . . . . . . . 60 10 Message Synchronize between Baetyl-Hub and Baidu IoTHub via Baetyl-Remote-MQTT module 69 10.1 Workflow . . . . . . . . . . . . . . . . . . . . . inference, device resources report etc. The combination of Baetyl and the Cloud Management Suite of BIE(Baidu IntelliEdge) will achieve cloud management and application distribution, enable applications running Application Production: Baetyl combines with Cloud Management Suite of BIE and many other productions of Baidu Cloud(such as CFC, Infinite, EasyEdge, TSDB, IoT Visualization) to provide data calcu- lation, storage

. . . . . . . . . . . . . . . . . . . . . . . 62 10 Message Synchronize between baetyl-hub and Baidu IoTHub via Remote Service 71 10.1 Workflow . . . . . . . . . . . . . . . . . . . . . . . . . . . inference, device resources report etc. The combination of Baetyl and the Cloud Management Suite of BIE(Baidu IntelliEdge) will achieve cloud management and application distribution, enable applications running Application Production: Baetyl combines with Cloud Management Suite of BIE and many other productions of Baidu Cloud(such as CFC, Infinite, EasyEdge, TSDB, IoT Visualization) to provide data calcu- lation, storage

所示： root@kali:~# fierce -dns baidu.com DNS Servers for baidu.com: ns2.baidu.com ns7.baidu.com dns.baidu.com ns3.baidu.com ns4.baidu.com Trying zone transfer first… Testing ns2.baidu.com Request timed out or transfer not allowed. Testing ns7.baidu.com Request timed out or transfer not allowed. Testing dns.baidu.com timed out or transfer not allowed. Testing ns3.baidu.com Request timed out or transfer not allowed. Testing ns4.baidu.com Request timed out or transfer not

text plurals differ Using custom modules and classes Machine translation Amagama Apertium AWS Baidu API machine translation DeepL Glosbe Google Translate Microsoft Cognitive Services Translator Microsoft MT_SERVICES MT_APERTIUM_APY MT_AWS_ACCESS_KEY_ID MT_AWS_SECRET_ACCESS_KEY MT_AWS_REGION MT_BAIDU_ID MT_BAIDU_SECRET MT_DEEPL_KEY MT_GOOGLE_KEY MT_MICROSOFT_COGNITIVE_KEY MT_MYMEMORY_EMAIL MT_MYMEMORY_KEY com/documentation/translate/] Baidu API machine translation New in version 3.2. Machine translation service provided by Baidu. This service uses an API and you need to obtain ID and API key from Baidu. To enable this

text plurals differ Using custom modules and classes Machine translation Amagama Apertium AWS Baidu API machine translation DeepL Glosbe Google Translate Microsoft Translator Microsoft Cognitive Services MT_SERVICES MT_APERTIUM_APY MT_AWS_ACCESS_KEY_ID MT_AWS_SECRET_ACCESS_KEY MT_AWS_REGION MT_BAIDU_ID MT_BAIDU_SECRET MT_DEEPL_KEY MT_GOOGLE_KEY MT_MICROSOFT_ID MT_MICROSOFT_SECRET MT_MICROSOFT_COGNITIVE_KEY com/documentation/translate/] Baidu API machine translation New in version 3.2. Machine translation service provided by Baidu. This service uses an API and you need to obtain ID and API key from Baidu. To enable this

text plurals differ Using custom modules and classes Machine translation Amagama Apertium AWS Baidu API machine translation DeepL Glosbe Google Translate Microsoft Cognitive Services Translator Microsoft MT_SERVICES MT_APERTIUM_APY MT_AWS_ACCESS_KEY_ID MT_AWS_SECRET_ACCESS_KEY MT_AWS_REGION MT_BAIDU_ID MT_BAIDU_SECRET MT_DEEPL_KEY MT_GOOGLE_KEY MT_MICROSOFT_COGNITIVE_KEY MT_MYMEMORY_EMAIL MT_MYMEMORY_KEY com/documentation/translate/] Baidu API machine translation New in version 3.2. Machine translation service provided by Baidu. This service uses an API and you need to obtain ID and API key from Baidu. To enable this