Ethan Rose Ozone PMC, committer 20 © 2022 Cloudera, Inc. All rights reserved. OZONE NAMESPACE LAYOUT /volume/bucket/key • Volume – Top level namespace grouping – Must have unique names – Can only BUCKET LAYOUT TYPES History • Ozone was originally built as a object store • Directories were simulated with prefixes 23 © 2022 Cloudera, Inc. All rights reserved. BEFORE BUCKET LAYOUT TYPES Problems filesystem.paths • No directories, only prefixes • Existing buckets automatically inherit this layout 25 © 2022 Cloudera, Inc. All rights reserved. NEW: OBJECT STORE BUCKETS (OBS) • Strict S3 compatibility

对象存储：采⽤ KV ⽅式管理对象元数据，⽆ 需管理元数据之间的关系 ⽂件系统：额外地，需要采⽤树结构作为索 引，管理元数据之间的关系 15 新存储格式 16 引⼊Bucket Metadata Layout I. FILE_SYSTEM_OPTIMIZED (FSO) : ⽀持纯粹的⽂件语义, 有限的 S3 兼容性 ⽂件的存储Key格式: “/” 在Bucket 创建时指定Layout，后续不⽀持更改 17 Benchmark Query Details: Dropped “catelog_sales” table with sub- paths(files/dirs) count = 5K Query Completion Time (in sec) HDFS 0.572 Ozone With FSO Layout 0.854 Ozone Ozone With OBS Layout 12.219 Hive 删除表(Rename操作) ● FileSystem delete on table directory path ● Moves table data to trash 举例: fs.delete(“/catelog_sales”) 18 合并Container RocksDB实例 - 现状和问题

Metadata Layout 版本号 ⽂件系统优化 ⽂件系统优化效果 Query Details: Dropped “catelog_sales” table with sub- paths(files/dirs) count = 5K Query Completion Time (in sec) HDFS 0.572 Ozone With FSO Layout 0.854 Ozone Ozone With OBS Layout 12.219 Hive 删除表(Rename操作) ● FileSystem delete on table directory path ● Moves table data to trash 举例: fs.delete(“/catelog_sales”) 均衡器Ozone Balancer(HDDS-4656) 时机