Stream Processing and Analytics Vasiliki (Vasia) Kalavri
 vkalavri@bu.edu Spring 2020 4/23: Cardinality and frequency estimation ??? Vasiliki Kalavri | Boston University 2020 Counting distinct elements University 2020 LogLog algorithm Input: stream S, array of m counters, hash fiction h Output: cardinality of S for j=0 to m-1 do: COUNT[j] = 0 for x in S do: i = h(x) j = getLeftBits(i, p) r = on Algorithms, 2003. • Flajolet, Philippe, et al. Hyperloglog: the analysis of a near-optimal cardinality estimation algorithm. 2007. https://hal.archives-ouvertes.fr/file/index/docid/406166/ filename/FlFuGaMe07

transparent. Referenced Cardinality The foreign key/relation cardinality of the referenced (parent) table or entity. Referencing Cardinality The foreign key/relation cardinality of the referencing (child) Designer. Cardinality on table_name1 Set the cardinality on table_name1: None, One and Only One, Many, One or Many, Zero or One, Zero or Many. Cardinality on table_name2 Set the cardinality on table_name2: Cardinality on entity_name1 Set the cardinality on entity_name1: None, One and Only One, Many, One or Many, Zero or One, Zero or Many. Cardinality on entity_name2 Set the cardinality on

color of the shape's border. Cap Style The cap style of the line/arrow. Cardinality The foreign key/relation cardinality of the table/entity. Case Sensitive The case sensitivity of the table or Designer. Cardinality on table_name1 Set the cardinality on table_name1: None, One and Only One, Many, One or Many, Zero or One, Zero or Many. Cardinality on table_name2 Set the cardinality on table_name2: Entity Designer. Cardinality on Set the cardinality on entity_name1: None, One and Only One, Many, 44 entity_name1 One or Many, Zero or One, Zero or Many. Cardinality on entity_name2

Table），并且必须指定聚合类型为 HLL_UNION。 HLL 类型主要用于非精确快速去重场景下，对数据进行预聚合。 HLL列只能通过配套的 hll_union_agg、hll_cardinality、hll_hash 进行查询或使用。 BITMAP 数据类型 BITMAP 数据类型 BITMAP 类型是一个二进制类型。BITMAP 类型只能用于聚合类型的表（Aggregation Table），并且必须指定聚合类型为 本文档主要介绍 HLL(HyperLogLog) 类型相关的内置函数。 标量函数 1.hll_cardinality 2.hll_hash 3.hll_empty 聚合函数 1.hll_union,hll_raw_agg 2.hll_union_agg HLL_CARDINALITY Description Description 功能：计算单个HLL类型值的基数。 返回值：bigint bitmap intersect_count, bitmap bigint bigint hll_cardinality hll_cardinality((hll a hll a)) mysql mysql>> select select hll_cardinality hll_cardinality((v1 v1)) from from tbl tbl;; ++-----------------------+

message. ③ Declare the expected response. The interaction type is determined implicitly from the cardinality of the input and output. The above example is a Request-Stream because one value is sent and a an @MessageMapping method supports is determined from the cardinality of the input (i.e. @Payload argument) and of the output, where cardinality means the following: Cardinali ty Description 1 Either The table below shows all input and output cardinality combinations and the corresponding interaction type(s): Input Cardinality Output Cardinality Interaction Types 0, 1 0 Fire-and-Forget, Request-Response

transparent. Referenced Cardinality The foreign key/relation cardinality of the referenced (parent) table or entity. Referencing Cardinality The foreign key/relation cardinality of the referencing (child) Cardinality on table_name1 Set the cardinality on table_name1: None, One and Only One, Many, One or Many, Zero or One, Zero or Many. Cardinality on table_name2 Set the cardinality on table_name2: Cardinality on entity_name1 Set the cardinality on entity_name1: None, One and Only One, Many, One or Many, Zero or One, Zero or Many. Cardinality on entity_name2 Set the cardinality on

color of the shape's border. Cap Style The cap style of the line/arrow. Cardinality The foreign key/relation cardinality of the table/entity. Case Sensitive The case sensitivity of the table or chosen. Cardinality on table_name1 Set the cardinality on table_name1: None, One and Only One, Many, One or Many, Zero or One, Zero or Many. Cardinality on table_name2 Set the cardinality on table_name2: Cardinality on entity_name1 Set the cardinality on entity_name1: None, One and Only One, Many, One or Many, Zero or One, Zero or Many. Cardinality on entity_name2 Set the cardinality on

color of the shape's border. Cap Style The cap style of the line/arrow. Cardinality The foreign key/relation cardinality of the table/entity. Case sensitive The case sensitivity of the table or chosen. Cardinality on table_name1 Set the cardinality on table_name1: None, One and Only One, Many, One or Many, Zero or One, Zero or Many. Cardinality on table_name2 Set the cardinality on table_name2: Cardinality on entity_name1 Set the cardinality on entity_name1: None, One and Only One, Many, One or Many, Zero or One, Zero or Many. Cardinality on entity_name2 Set the cardinality on

message. ③ Declare the expected response. The interaction type is determined implicitly from the cardinality of the input and output. The above example is a Request-Stream because one value is sent and a @MessageMapping method supports is determined from the 172 cardinality of the input (i.e. @Payload argument) and of the output, where cardinality means the following: Cardinali ty Description 1 Either The table below shows all input and output cardinality combinations and the corresponding interaction type(s): Input Cardinality Output Cardinality Interaction Types 0, 1 0 Fire-and-Forget, Request-Response

considerations: • What are the highest QPS queries, and what are the where clauses for them? • Cardinality of the column; it must be high. • Do we want some rows to live together to support in-shard joins clause uses customer_id. So, it shall have a Primary Vindex. • Given that it has lots of users, its cardinality is also high. • For the corder table, we have a choice between customer_id and order_id. Given map to a tenant_id. In such cases, the cardinality of a tenant id may be too low. It’s also common that such systems have queries that use other high cardinality columns in their where clauses. Those should