Go runtime related problems in TiDB production environment About me ● Arthur Mao(毛康力), Senior Engineer@PingCAP ● TiDB core developer (top3 contributor) ● GitBook about golang internals (@tiancaiamao) IO is ready => goroutine wake up == 4.3ms ○ Sometime even 10ms+ latency here! ○ The time spend on runtime schedule is not negligible ● When CPU is overload, which goroutine should be given priority? Analysis longer to be scheduled ● The runtime scheduling does not consider priority ● CPU dense workload could affect IO latency Conclusion Part II - Memory control ● Go Runtime ○ Allocated from OS (mmaped)

浅谈全链路监控： 从应用到数据库到 Runtime 黄东旭, Co-founder & CTO, PingCAP 关于我 黄东旭，联合创始人 & CTO @ PingCAP 做分布式数据库的程序员 ● 现在能写代码的时间是奢侈品 TiDB 的亲爹之一兼首席客服和新功能的第一个用户 ● 冤有头债有主，SQL 慢了来找我。。。 偶尔玩玩音乐 ● 摇滚乐->实验音乐 Go 的粉丝！！！！ tool trace go tool trace ● 优点：好用，好看（UI） ● 缺点：性能损耗太大，不能一直开着 Trace in Go runtime ● go tool trace 的原理是？ Trace 会 Go Runtime 的代码中打桩收集 CPU time，在 Goroutine 开始执行时记录 start_run_time, 在调度退出执行时记录 end_run_time，累加 goroutine 的 CPU time。 A little bit about Go runtime https://learnku.com/articles/41728 https://github.com/golang/go/blob/ma ster/src/runtime/trace.go hack runtime 的思路： follow the tracing event. PingCAP

............................................................................ 17 Microsoft C/C++ Runtime Libraries ...................................................................................... Mixed usage of the NONE and other character sets is not recommended as it can lead to to unexpected runtime errors. CHECK Constraint Change Formerly, CHECK constraints were not SQL standard-compliant in regard particularly for Windows Embedded deployments. Please refer to the later section entitled Microsoft Runtime Libraries. Installation Choices On Windows, you have three server models to choose from: Superserver

Using the Client and Loader Tools 662 Prerequisites 662 Setting Up Your Greenplum Database Clients Runtime Environment 663 Running the Client and Loader Programs 663 Greenplum Database Documentation References Parameters 1880 Greenplum Performance Database 1880 Query Metrics Collection Parameters 1880 Runtime Statistics Collection Parameters 1881 Automatic Statistics Collection Parameters 1881 Client Connection avaialble memory. This could lead to File System Full errors during filesystem checks. 32446 Fixed runtime errors that could occur when running gpcheckperf with the -V flag (very verbose mode). Now, when

Using the Client and Loader Tools 655 Prerequisites 655 Setting Up Your Greenplum Database Clients Runtime Environment 655 Running the Client and Loader Programs 656 Greenplum Database Documentation References Parameters 1846 Greenplum Performance Database 1846 Query Metrics Collection Parameters 1846 Runtime Statistics Collection Parameters 1847 Automatic Statistics Collection Parameters 1847 VMware Greenplum avaialble memory. This could lead to File System Full errors during filesystem checks. 32446 Fixed runtime errors that could occur when running gpcheckperf with the -V flag (very verbose mode). Now, when

Using the Client and Loader Tools 633 Prerequisites 633 Setting Up Your Greenplum Database Clients Runtime Environment 634 Running the Client and Loader Programs 634 Greenplum Database Documentation References Parameters 1840 Greenplum Performance Database 1840 Query Metrics Collection Parameters 1840 Runtime Statistics Collection Parameters 1840 Automatic Statistics Collection Parameters 1840 Client Connection avaialble memory. This could lead to File System Full errors during filesystem checks. 32446 Fixed runtime errors that could occur when running gpcheckperf with the -V flag (very verbose mode). Now, when

Using the Client and Loader Tools 631 Prerequisites 631 Setting Up Your Greenplum Database Clients Runtime Environment 632 Running the Client and Loader Programs 632 Greenplum Database Documentation References Parameters 1787 Greenplum Performance Database 1787 Query Metrics Collection Parameters 1787 Runtime Statistics Collection Parameters 1787 Automatic Statistics Collection Parameters 1787 Client Connection Resolved Issues VMware Tanzu Greenplum 6.19.1 resolves these issues: Server 31957, 12930 Resolves a runtime performance issue that was observed when a query that included a NOT IN / IN construct was more

Using the Client and Loader Tools 539 Prerequisites 539 Setting Up Your Greenplum Database Clients Runtime Environment 539 Running the Client and Loader Programs 540 VMware Tanzu Greenplum 6 Documentation Parameters 1570 Greenplum Performance Database 1570 Query Metrics Collection Parameters 1571 Runtime Statistics Collection Parameters 1571 Automatic Statistics Collection Parameters 1571 Client Connection Resolved Issues VMware Tanzu Greenplum 6.19.1 resolves these issues: Server 31957, 12930 Resolves a runtime performance issue that was observed when a query that included a NOT IN / IN construct was more

Using the Client and Loader Tools 629 Prerequisites 629 Setting Up Your Greenplum Database Clients Runtime Environment 630 Running the Client and Loader Programs 630 Greenplum Database Documentation References Parameters 1778 Greenplum Performance Database 1779 Query Metrics Collection Parameters 1779 Runtime Statistics Collection Parameters 1779 Automatic Statistics Collection Parameters 1779 Client Connection Resolved Issues VMware Tanzu Greenplum 6.19.1 resolves these issues: Server 31957, 12930 Resolves a runtime performance issue that was observed when a query that included a NOT IN / IN construct was more

Using the Client and Loader Tools 457 Prerequisites 457 Setting Up Your Greenplum Database Clients Runtime Environment 458 Running the Client and Loader Programs 458 Greenplum Database Documentation References 1699 What to Log 1699 Automatic Vacuum Parameters 1700 Query Metrics Collection Parameters 1700 Runtime Statistics Collection Parameters 1700 VMware Greenplum 7 Documentation VMware by Broadcom 99 Automatic cannot use resource groups to manage and limit the total CPU and memory resources for a PL/Container runtime. Container instances are limited only by system resources, and the containers may consume resources