rec 100 recs Intuition: use the dataflow graph to extract operator dependencies and system instrumentation to collect accurate, representative metrics. target: 40 rec/s 0.5s 15 ??? Vasiliki Kalavri rec 100 recs Intuition: use the dataflow graph to extract operator dependencies and system instrumentation to collect accurate, representative metrics. x4 instances to keep up with src rate target: rec 100 recs Intuition: use the dataflow graph to extract operator dependencies and system instrumentation to collect accurate, representative metrics. True rate = 200 recs/s x4 instances to keep

based on Prometheus + Grafana + EFK ● Ingest metrics and logs into Stackdriver without any instrumentation changes ● Aggregate logs from many clusters -- whether GKE or GKE On-Prem Logging and Monitoring

Kyuubi - Server Metrics • Trouble Shooting • Spark Online Documentation – Monitoring and Instrumentation 6.6.2 Monitoring Kyuubi - Server Metrics Kyuubi has a configurable metrics system based on users to report Kyuubi metrics to a variety of kyuubi.metrics.reporters. The metrics provide instrumentation for specific activities and Kyuubi server. Configurations The metrics system is configured

Kyuubi - Server Metrics • Trouble Shooting • Spark Online Documentation – Monitoring and Instrumentation 6.6.2 Monitoring Kyuubi - Server Metrics Kyuubi has a configurable metrics system based on users to report Kyuubi metrics to a variety of kyuubi.metrics.reporters. The metrics provide instrumentation for specific activities and Kyuubi server. Configurations The metrics system is configured

Kyuubi - Server Metrics • Trouble Shooting • Spark Online Documentation – Monitoring and Instrumentation 6.6.2 Monitoring Kyuubi - Server Metrics Kyuubi has a configurable metrics system based on users to report Kyuubi metrics to a variety of kyuubi.metrics.reporters. The metrics provide instrumentation for specific activities and Kyuubi server. 6.6. Monitoring 123 Kyuubi, Release 1.5.0-incubating

Monitoring Kyuubi - Server Metrics Trouble Shooting Spark Online Documentation Monitoring and Instrumentation [http://spark.apache.org/docs/latest/monitoring.html] 2. Monitoring Kyuubi - Server Metrics users to report Kyuubi metrics to a variety of kyuubi.metrics.reporters. The metrics provide instrumentation for specific activities and Kyuubi server. 2.1. Configurations The metrics system is configured

Kyuubi - Server Metrics • Trouble Shooting • Spark Online Documentation – Monitoring and Instrumentation 6.6.2 Monitoring Kyuubi - Server Metrics Kyuubi has a configurable metrics system based on users to report Kyuubi metrics to a variety of kyuubi.metrics.reporters. The metrics provide instrumentation for specific activities and Kyuubi server. 6.6. Monitoring 123 Kyuubi, Release 1.5.1-incubating

Monitoring Kyuubi - Server Metrics Trouble Shooting Spark Online Documentation Monitoring and Instrumentation [http://spark.apache.org/docs/latest/monitoring.html] 2. Monitoring Kyuubi - Server Metrics users to report Kyuubi metrics to a variety of kyuubi.metrics.reporters. The metrics provide instrumentation for specific activities and Kyuubi server. 2.1. Configurations The metrics system is configured

Kyuubi - Server Metrics • Trouble Shooting • Spark Online Documentation – Monitoring and Instrumentation 6.6.2 Monitoring Kyuubi - Server Metrics Kyuubi has a configurable metrics system based on users to report Kyuubi metrics to a variety of kyuubi.metrics.reporters. The metrics provide instrumentation for specific activities and Kyuubi server. 6.6. Monitoring 123 Kyuubi, Release 1.5.2-incubating

Kyuubi - Server Metrics • Trouble Shooting • Spark Online Documentation – Monitoring and Instrumentation 3.4. Monitoring 77 Kyuubi, Release 1.7.0 3.4.2 Monitoring Kyuubi - Server Metrics Kyuubi users to report Kyuubi metrics to a variety of kyuubi.metrics.reporters. The metrics provide instrumentation for specific activities and Kyuubi server. Configurations The metrics system is configured