appendBuffer ... ... closeThreadPool (close these log writers) getChunkToWrite Write the largest chunk Close all writers at the end Region 1 Region 2 Region 3 Region 4 Old design of log splitting Log WriterAndPath writeThenClose WriterAndPath writeThenClose getChunkToWrite Write the largest chunk whose size exceeds maxHeapUsage After reading all entries into memory WriterAndPath writeAndClose

almost all of the HFile Block are about 64KB, so it would be a SingleByteBuff because our ByteBuffer chunk is 4MB in ByteBufferArray. The above case#2 is a rare case though higher cost. #3 Block decompression

• 本地化读写太差，网络IO消耗严重 43 Postgres Conference China 2016 中国用户大会 HBase客户端性能优化 • 使用批量数据处理接口 • 保持2MB的Chunk Size • 使用内存pool缓存HTable及其他可重用对象 • 使用多线程并发技术 – Parallel Scanner • 使用异步调用接口 – AsyncClient • 使用数据预取以及预缓存