Real-Time Unified Data Layers: A New Era for Scalable Analytics, Search, and AI v 1.1Table of Contents Introduction 1. The Interconnection of Analytics, Search, and AI 2. What is a Real-Time Unified Data Data Layer? 3. Why Do You Need a Real-Time Unified Data Layer? 4. 5.CrateDB: A Modern Real-Time Unified Data Layer1. Introduction Data teams are facing more challenges than ever. As applications generate and architecture teams must rethink traditional data infrastructures. The future lies in Real-Time Unified Data Layers—platforms that seamlessly support analytics, search, and AI workloads at scale. These

Go runtime related problems in TiDB production environment About me ● Arthur Mao(毛康力), Senior Engineer@PingCAP ● TiDB core developer (top3 contributor) ● GitBook about golang internals (@tiancaiamao) IO is ready => goroutine wake up == 4.3ms ○ Sometime even 10ms+ latency here! ○ The time spend on runtime schedule is not negligible ● When CPU is overload, which goroutine should be given priority? Analysis longer to be scheduled ● The runtime scheduling does not consider priority ● CPU dense workload could affect IO latency Conclusion Part II - Memory control ● Go Runtime ○ Allocated from OS (mmaped)

浅谈全链路监控： 从应用到数据库到 Runtime 黄东旭, Co-founder & CTO, PingCAP 关于我 黄东旭，联合创始人 & CTO @ PingCAP 做分布式数据库的程序员 ● 现在能写代码的时间是奢侈品 TiDB 的亲爹之一兼首席客服和新功能的第一个用户 ● 冤有头债有主，SQL 慢了来找我。。。 偶尔玩玩音乐 ● 摇滚乐->实验音乐 Go 的粉丝！！！！ tool trace go tool trace ● 优点：好用，好看（UI） ● 缺点：性能损耗太大，不能一直开着 Trace in Go runtime ● go tool trace 的原理是？ Trace 会 Go Runtime 的代码中打桩收集 CPU time，在 Goroutine 开始执行时记录 start_run_time, 在调度退出执行时记录 end_run_time，累加 goroutine 的 CPU time。 A little bit about Go runtime https://learnku.com/articles/41728 https://github.com/golang/go/blob/ma ster/src/runtime/trace.go hack runtime 的思路： follow the tracing event. PingCAP

elastic scaling, encryption features & more. The project is committed to providing a multi‐source heterogeneous, enhanced database platform and further building an ecosystem around the upper layer of the platform Apache ShardingSphere, aims at building the standard and ecosystem on the upper layer of the heterogeneous database. It focuses on how to make full and reasonable use of the computing and storage capabilities database proxy, providing a database server that encapsulates database binary protocol to support heterogeneous languages. 1 Apache ShardingSphere document 1.2 Product Features F ea tu re Definition

scheduling, and opti- mize the scheduling formula to prevent the bursting issue (which is similar to heterogeneous space clusters) caused by large compression rate discrepancies #3592 #10005 • Tools – Backup interruption caused by the failure of a few TiKV nodes #980 – TiCDC * Fix the concurrency issue in Unified Sorter and filter the unhelpful error messages #1678 * Fix a bug that the creation of redundant zone information is lost in the Avro output #1712 * Support cleaning up stale temporary files in Unified Sorter and forbid sharing the sort-dir directory #1742 44 * Fix a deadlock bug in the KV client

protocol, SQL dialect and database storage. It can quickly link applications and multi‐mode heterogeneous databases quickly. • Enhance: Capture database access entry to provide additional features transparently and distributed governance, applicable in a variety of situations such as Java isomorphism, heterogeneous language and Cloud‐Native. As the cornerstone of enterprises, the relational database has a huge database proxy, providing a database server that encapsulates database binary protocol to support heterogeneous languages. Currently, MySQL and PostgreSQL (compatible with PostgreSQL‐based databases, such

278 Rule Config . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279 Runtime Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279 5 Dev Manual and distributed governance, applicable in a variety of situa‐ tions such as Java isomorphism, heterogeneous language and cloud native. Apache ShardingSphere aiming at reasonably making full use of the database proxy, providing a database server that encapsulates database binary protocol to support heterogeneous languages. Friendlier to DBA, the MySQL version provided now can use any kind of terminal (such

raftdb.allow-concurrent-memtable-write: true server.grpc-concurrency: 6 readpool.unified.min-thread-count: 5 50 readpool.unified.max-thread-count: 20 readpool.storage.normal-concurrency: 10 pessimistic-txn raftstore.store-pool-size: 3 readpool.storage.normal-concurrency: 10 readpool.unified.max-thread-count: 20 readpool.unified.min-thread-count: 5 rocksdb.max-background-jobs: 8 server.grpc-concurrency: 6 server_configs: tidb: log.slow-threshold: 300 tikv: readpool.storage.use-unified-pool: false readpool.coprocessor.use-unified-pool: true pd: replication.enable-placement-rules: true replication.location-labels:

raftdb.allow-concurrent-memtable-write: true server.grpc-concurrency: 6 readpool.unified.min-thread-count: 5 readpool.unified.max-thread-count: 20 readpool.storage.normal-concurrency: 10 pessimistic-txn.pipelined: raftstore.store-pool-size: 3 readpool.storage.normal-concurrency: 10 readpool.unified.max-thread-count: 20 readpool.unified.min-thread-count: 5 rocksdb.max-background-jobs: 8 server.grpc-concurrency: 6 server_configs: tidb: log.slow-threshold: 300 tikv: readpool.storage.use-unified-pool: false readpool.coprocessor.use-unified-pool: true pd: replication.enable-placement-rules: true replication.location-labels:

is positioned as a Database Plus, and aims at building a standard layer and ecosystem above heterogeneous databases. It focuses on how to reuse existing databases and their respective upper layer, rather database protocol, SQL dialect and database storage. It can quickly connect applications and heterogeneous databases quickly. • Enhance: Capture database access entry to provide additional features transparently and distributed governance, applicable in a variety of situations such as Java isomorphism, heterogeneous language and Cloud‐Native. As the cornerstone of enterprises, the relational database has a huge