recovery semantics, and guarantees Vasiliki Kalavri | Boston University 2020 Today’s topics • High-availability and fault-tolerance in distributed stream processing • Recovery semantics and guarantees positive. Vasiliki Kalavri | Boston University 2020 Further resources • Jeong-Hyon Hwang et al. High-Availability Algorithms for Distributed Stream Processing. (ICDE ’05). • http://cs.brown.edu/research/aurora/hwang

Stream Processing and Analytics Vasiliki (Vasia) Kalavri
 vkalavri@bu.edu Spring 2020 3/31: High-availability & reconfiguration ??? Vasiliki Kalavri | Boston University 2020 • To recover from failures state 2 Checkpointing guards the state from failures, but what about process failure? High-availability ??? Vasiliki Kalavri | Boston University 2020 3 Flink processes ??? Vasiliki Kalavri | Boston keeps metadata about application execution, such as pointers to completed checkpoints. • A high-availability mode migrates the responsibility and metadata for a job to another JobManager in case the

Store • Relational Tables • JSON Documents • X Protocol • X DevAPI Replication & High-Availability • InnoDB Cluster, ClusterSet • Group Replication • Router • Built-in Replication

and Offline installation ● Private container registry support ● Latest 3 versions of k8s ● High-availability control plane ● Auto-repair Installation and Configuration $ gke-on-prem create cluster --dry-run

Retractions & results amendment Reconfiguration & updates Debugging Fault-tolerance & high-availability Vasiliki Kalavri | Boston University 2020 actions, alerts continuous analytics … Building

the query is running might be impractical. • state accumulation and re-partitioning • high-availability and low latency requirements • scheduling overhead Challenges in streaming optimization

splitting functionality, support for the Kubernetes Service Mesh Interface (SMI), graduates high-availability support out of experimental status, and adds a tremendous list of other improvements, performance

machines, each offering local computation and storage. Rather than rely on hardware to deliver high-availability, the library itself is designed to detect and handle failures at the application layer, so delivering

in production builds Improves failure diagnostics in checked builds. Provides resilience in high-availability systems.Conclusion • Articulate the contracts for your functions, even those parts that cannot

and then SUSE Rancher server is installed using the Helm package manager for Kubernetes. For high-availability, it is recommended to have a three-node Kubernetes cluster. Ensure that you are aware of the