Schemes ❏ Should be able to recover the files automatically on failures ❏ Online recovery ❏ Phase - II ❏ Offline recovery ❏ Phase - III ❏ Should provide options to enable EC via Recon / CM ❏ Should decode operation. ➢ To avoid the degraded reads, we need to recover the lost replicas offline. EC Recovery Reads ReplicaIndex-1 ReplicaIndex-2 ReplicaIndex-3 ReplicaIndex-4 ReplicaIndex-5 EC Container Reconstruct and Read stripe-1 Reconstruct and Read stripe-2 N1 N2 N3 N4 N5 What is the Offline Recovery? ➢ When a node/Disk lost, we will lose the containers which are residing in that node/disk. We

Chunks Blocks User Chunks Chunks Blocks Containers Chunks Chunks Containers Volumes Unit of Recovery Unit of Failure Storage Container Manager Chunks Chunks Volumes Datanodes Why does foreground 600TB per node 1000’s of nodes at 100TB per node Scalability 10B Objects 400M Objects Recovery Fast recovery ( < 5 min restart) Slow startup based on size High Availability Active - Active Active

Chunks Blocks User Chunks Chunks Blocks Containers Chunks Chunks Containers Volumes Unit of Recovery Unit of Failure Storage Container Manager Chunks Chunks Volumes Datanodes AGGREGATION VIA CONTAINERS 600TB per node 1000’s of nodes at 100TB per node Scalability 10B Objects 400M Objects Recovery Fast recovery Slow startup based on size High Availability Active - Active Active - Standby Protocol