Scalable Stream Processing - Spark Streaming and Flinkcomputation as a series of very small, deterministic batch jobs. • Chops up the live stream into batches of X seconds. • Treats each batch as RDDs and processes them using RDD operations. • Discretized computation as a series of very small, deterministic batch jobs. • Chops up the live stream into batches of X seconds. • Treats each batch as RDDs and processes them using RDD operations. • Discretized computation as a series of very small, deterministic batch jobs. • Chops up the live stream into batches of X seconds. • Treats each batch as RDDs and processes them using RDD operations. • Discretized0 码力 | 113 页 | 1.22 MB | 1 年前3- Streaming optimizations - CS 591 K1: Data Stream Processing and Analytics Spring 2020replication • Parallel recovery mechanism in case of failures 44 input stream time-based micro-batches D-Streams • During an interval, input data received is stored using RDDs • A D-Stream is a group0 码力 | 54 页 | 2.83 MB | 1 年前3
共 2 条
- 1