vkalavri@bu.edu Spring 2020 4/21: Sampling and filtering streams ??? Vasiliki Kalavri | Boston University 2020 Synopses for massive data streams • Maintaining synopses is often the only means of providing providing interactive response times when exploring massive datasets or high speed data streams. • Queries are executed against the synopsis rather than the entire dataset. 2 Synopsis: a lossy, compact the exact answer is $5,001,482.76. 4 ??? Vasiliki Kalavri | Boston University 2020 Sampling streams 5 ??? Vasiliki Kalavri | Boston University 2020 6 A sample is a set of data elements selected

incrementally over time, rather than being available in full before its processing begins. • Data streams are high-volume, real-time data that might be unbounded • we cannot store the entire stream in on-the-fly using limited memory 2 Vasiliki Kalavri | Boston University 2020 Properties of data streams • They arrive continuously instead of being available a-priori. • They bear an arrival and/or Streaming Data Warehouse • low-latency materialized view updates • pre-aggregated, pre-processed streams and historical data Data Management Approaches 4 storage analytics static data streaming data

Transforming languages define transformations specifying operations that process input streams and produce output streams. • Declarative languages specify the expected results of the computation rather than than the execution flow. • Imperative languages are used to describe plans of operators the streams must flow through. • Pattern-based languages specify conditions and actions to be taken when stream-to-relation: define tables by selecting portions of a stream. • relation-to-stream: create streams through querying tables Declarative language: CQL 4 Vasiliki Kalavri | Boston University 2020

on all 64bit builds. • Buffers of data use unsigned chars (or uint8_t). • Offsets within files/streams are represented using int64_t. In addition, we use floats (and avoid doubles when possible), assumed all streams (except compress-fonts/images). • compress Compress all streams. • compress-fonts Compress embedded fonts. • compress-images Compress images. • ascii ASCII hex encode binary streams. • for web browsers. • clean Pretty-print graphics commands in content streams. • sanitize Sanitize graphics commands in content streams. • incremental Write changes as incremental update. • continue-on-error

part of the documentation covers support for reactive-stack web applications built on a Reactive Streams API to run on non-blocking servers, such as Netty, Undertow, and Servlet 3.1+ containers. Individual framework, Spring WebFlux, was added later in version 5.0. It is fully non-blocking, supports Reactive Streams back pressure, and runs on such servers as Netty, Undertow, and Servlet 3.1+ containers. Both web control the rate of events so that a fast producer does not overwhelm its destination. Reactive Streams is a small spec (also adopted in Java 9) that defines the interaction between asynchronous components

on all 64bit builds. • Buffers of data use unsigned chars (or uint8_t). • Offsets within files/streams are represented using int64_t. In addition, we use floats (and avoid doubles when possible), assumed all streams (except compress-fonts/images). • compress Compress all streams. • compress-fonts Compress embedded fonts. • compress-images Compress images. • ascii ASCII hex encode binary streams. • for web browsers. • clean Pretty-print graphics commands in content streams. • sanitize Sanitize graphics commands in content streams. • incremental Write changes as incremental update. • continue-on-error

on all 64bit builds. • Buffers of data use unsigned chars (or uint8_t). • Offsets within files/streams are represented using int64_t. In addition, we use floats (and avoid doubles when possible), assumed PDFObject In exception to using the MuPDF coordinate space, if we are dealing directly with PDFObject streams then we are using the PDF coordinate space. Therefore if we doing something low-level like adding all streams (except compress-fonts/images). • compress Compress all streams. • compress-fonts Compress embedded fonts. • compress-images Compress images. • ascii ASCII hex encode binary streams. •

on all 64bit builds. • Buffers of data use unsigned chars (or uint8_t). • Offsets within files/streams are represented using int64_t. In addition, we use floats (and avoid doubles when possible), assumed all streams (except compress-fonts/images). • compress Compress all streams. • compress-fonts Compress embedded fonts. • compress-images Compress images. • ascii ASCII hex encode binary streams. • for web browsers. • clean Pretty-print graphics commands in content streams. • sanitize Sanitize graphics commands in content streams. • incremental Write changes as incremental update. • continue-on-error

pressed and 'end' event when ctrl-d is pressed process.stdout and process.stderr - blocking, writable streams important so output from asynchronous functions isn’t interspersed process.title - get/set name Assertion Testing - 38 OS - 41 Readline - 44 TTY - 48 Events - 50 Path - 54 File System - 56 Streams - 68 Zlib - 74 String Decoder - 76 Net - 77 Datagram (UDP) - 86 DNS - 91 HTTP - 93 URL - 97 network interface hardware, reducing load on network resources. Core Modules Readline ... Reads streams one line at a time Supports many control keys when reading from stdin see _ttwrite method in readline

(7) streams (8) type erasurechapters (1) motivating futures/promises + actors (2) libprocess (3) revisiting the problem (4) evolution of libprocess (5) eventuals (6) scheduling (7) streams (8) (7) streams (8) type erasurechapters (1) motivating futures/promises + actors (2) libprocess (3) revisiting the problem (4) evolution of libprocess (5) eventuals (6) scheduling (7) streams (8) (7) streams (8) type erasurechapters (1) motivating futures/promises + actors (2) libprocess (3) revisiting the problem (4) evolution of libprocess (5) eventuals (6) scheduling (7) streams (8)