configuration item is successfully modified, the result is persisted in etcd instead of in the configuration file; the configuration in etcd will prevail in the subsequent operations. The names of some configuration ums • Alert rule: (sum(pd_cluster_status{type="store_down_count"})by (instance)> 0)and ( �→ sum(etcd_server_is_leader)by (instance)> 0) • Description: PD has not received a TiKV/TiFlash heartbeat for it offline. 7.5.2.2 Critical-level alerts 340 7.5.2.2.1 PD_etcd_write_disk_latency • Alert rule: histogram_quantile(0.99, sum(rate(etcd_disk_wal_fsync_duration_seconds_bucket �→ [1m]))by (instance

configuration item is successfully modified, the result is persisted in etcd instead of in the configuration file; the configuration in etcd will prevail in the subsequent operations. The names of some configuration ums • Alert rule: (sum(pd_cluster_status{type="store_down_count"})by (instance)> 0)and ( �→ sum(etcd_server_is_leader)by (instance)> 0) • Description: PD has not received a TiKV/TiFlash heartbeat for it offline. 7.5.2.2 Critical-level alerts 371 7.5.2.2.1 PD_etcd_write_disk_latency • Alert rule: histogram_quantile(0.99, sum(rate(etcd_disk_wal_fsync_duration_seconds_bucket �→ [1m]))by (instance

configuration item is successfully modified, the result is persisted in etcd instead of in the configuration file; the configuration in etcd will prevail in the subsequent operations. The names of some configuration ums • Alert rule: (sum(pd_cluster_status{type="store_down_count"})by (instance)> 0)and ( �→ sum(etcd_server_is_leader)by (instance)> 0) • Description: PD has not received a TiKV/TiFlash heartbeat for it offline. 7.5.2.2 Critical-level alerts 354 7.5.2.2.1 PD_etcd_write_disk_latency • Alert rule: histogram_quantile(0.99, sum(rate(etcd_disk_wal_fsync_duration_seconds_bucket �→ [1m]))by (instance

设置成功会返回 Query OK： Query OK, 0 rows affected (0.01 sec) 针对 PD 可在线修改的配置项，成功修改后则会持久化到 etcd 中，不会对配置文件进行持久化，后续以 etcd 中 的配置为准。同上，若和 TiDB 预留关键字冲突，需要用反引号 ` 包裹此类配置项，例如 `schedule.leader- �→ schedule-limit`。 PD_cluster_down_store_nums • 报警规则： (sum(pd_cluster_status{type="store_down_count"})by (instance)> 0)and (sum(etcd_server_is_ �→ leader)by (instance)> 0) • 规则描述： PD 长时间（默认配置是 30 分钟）没有收到 TiKV/TiFlash 心跳。 • 处理方法： 2.2.1 PD_etcd_write_disk_latency • 报警规则： histogram_quantile(0.99, sum(rate(etcd_disk_wal_fsync_duration_seconds_bucket[1m]))by ( �→ instance, job, le))> 1 • 规则描述： fsync 操作延迟大于 1s，代表 etcd 写盘慢，这很容易引起

设置成功会返回 Query OK： Query OK, 0 rows affected (0.01 sec) 针对 PD 可在线修改的配置项，成功修改后则会持久化到 etcd 中，不会对配置文件进行持久化，后续以 etcd 中 的配置为准。同上，若和 TiDB 预留关键字冲突，需要用反引号 ` 包裹此类配置项，例如 `schedule.leader- �→ schedule-limit`。 PD_cluster_down_store_nums • 报警规则： (sum(pd_cluster_status{type="store_down_count"})by (instance)> 0)and (sum(etcd_server_is_ �→ leader)by (instance)> 0) • 规则描述： PD 长时间（默认配置是 30 分钟）没有收到 TiKV/TiFlash 心跳。 • 处理方法： 2.2.1 PD_etcd_write_disk_latency • 报警规则： histogram_quantile(0.99, sum(rate(etcd_disk_wal_fsync_duration_seconds_bucket[1m]))by ( �→ instance, job, le))> 1 • 规则描述： fsync 操作延迟大于 1s，代表 etcd 写盘慢，这很容易引起

configuration item is successfully modified, the result is persisted in etcd instead of in the configuration file; the configuration in etcd will prevail in the subsequent operations. The names of some configuration ums • Alert rule: (sum(pd_cluster_status{type="store_down_count"})by (instance)> 0)and ( �→ sum(etcd_server_is_leader)by (instance)> 0) • Description: PD has not received a TiKV/TiFlash heartbeat for it offline. 7.5.2.2 Critical-level alerts 465 7.5.2.2.1 PD_etcd_write_disk_latency • Alert rule: histogram_quantile(0.99, sum(rate(etcd_disk_wal_fsync_duration_seconds_bucket �→ [1m]))by (instance

设置成功会返回 Query OK： Query OK, 0 rows affected (0.01 sec) 针对 PD 可在线修改的配置项，成功修改后则会持久化到 etcd 中，不会对配置文件进行持久化，后续以 etcd 中 的配置为准。同上，若和 TiDB 预留关键字冲突，需要用反引号 ` 包裹此类配置项，例如 `schedule.leader- �→ schedule-limit`。 PD_cluster_down_store_nums • 报警规则： (sum(pd_cluster_status{type="store_down_count"})by (instance)> 0)and (sum(etcd_server_is_ �→ leader)by (instance)> 0) • 规则描述： PD 长时间（默认配置是 30 分钟）没有收到 TiKV/TiFlash 心跳。 • 处理方法： 2.2.1 PD_etcd_write_disk_latency • 报警规则： histogram_quantile(0.99, sum(rate(etcd_disk_wal_fsync_duration_seconds_bucket[1m]))by ( �→ instance, job, le))> 1 • 规则描述： fsync 操作延迟大于 1s，代表 etcd 写盘慢，这很容易引起

设置成功会返回 Query OK： Query OK, 0 rows affected (0.01 sec) 针对 PD 可在线修改的配置项，成功修改后则会持久化到 etcd 中，不会对配置文件进行持久化，后续以 etcd 中 的配置为准。同上，若和 TiDB 预留关键字冲突，需要用反引号 ` 包裹此类配置项，例如 `schedule.leader- �→ schedule-limit`。 PD_cluster_down_store_nums • 报警规则： (sum(pd_cluster_status{type="store_down_count"})by (instance)> 0)and (sum(etcd_server_is_ �→ leader)by (instance)> 0) • 规则描述： PD 长时间（默认配置是 30 分钟）没有收到 TiKV/TiFlash 心跳。 • 处理方法： 2.2.1 PD_etcd_write_disk_latency • 报警规则： histogram_quantile(0.99, sum(rate(etcd_disk_wal_fsync_duration_seconds_bucket[1m]))by ( �→ instance, job, le))> 1 • 规则描述： fsync 操作延迟大于 1s，代表 etcd 写盘慢，这很容易引起

configuration item is successfully modified, the result is persisted in etcd instead of in the configuration file; the configuration in etcd will prevail in the subsequent operations. The names of some configuration ums • Alert rule: (sum(pd_cluster_status{type="store_down_count"})by (instance)> 0)and ( �→ sum(etcd_server_is_leader)by (instance)> 0) • Description: PD has not received a TiKV/TiFlash heartbeat for it offline. 9.6.2.2 Critical-level alerts 916 9.6.2.2.1 PD_etcd_write_disk_latency • Alert rule: histogram_quantile(0.99, sum(rate(etcd_disk_wal_fsync_duration_seconds_bucket �→ [1m]))by (instance

might panic when gRPC threads are checking is_shutdown #16236 @pingyu • PD • Fix the issue that the etcd health check in PD does not remove expired addresses #7226 @iosmanthus • Fix the issue that when architecture As an incremental data replication tool for TiDB, TiCDC is highly available through PD’s etcd. TiCDC pulls data changes from TiKV nodes through multiple Capture pro- cesses, and then sorts and configuration item is successfully modified, the result is persisted in etcd instead of in the configuration file; the configuration in etcd will prevail in the subsequent operations. The names of some configuration