Model and Operate Datacenter by Kubernetes at eBay 辛肖刚, Cloud Engineering Manager, ebay 梅岑恺, Senior Operation Manager, ebay Agenda About ebay Our fleet Kubernetes makes magic at ebay Model + Controller Controller How we model our datacenter Operation in large scale Q&A About ebay 177M Active buyers worldwide $22.7B Amount of eBay Inc. GMV $2.6B Reported revenue 62% International revenue 1.1B Kubernetes Onboard Provision Configuration Kubernetes You need onboard something from nothing! Let’s model a datacenter running Kubernetes Onboard Provision Configuration Kubernetes After you define your

configuration) Orchestrate and manage on-prem containers just like GKE in the cloud Consistent operating model with access to GCP services across hybrid environments Single-pane-of-glass for multiple Kubernetes Control Plane Node GKE On-Prem Node Control Plane Node Hybrid Use Cases Legacy Software Local Execution Edge / IoT Cloud bursting Ecommerce site Catalog, ERP Warehouse Factory Branch Augmented Services

approach to enhance container isolation gVisor is special Machine-level virtualization Rule-based execution gVisor Technology landscape DEMO

Chen, Ant Financial TEE-based Secrets Protection: Solution Confidential Computing A Trusted Execution Environment (TEE) is • a secure area protected by the processor (aka. Enclave) Example: Intel

PyTorch, MXNet, Chainer, and more • JupyterHub to create and manage interactive Jupyter notebooks • Model serving – serve exported models with TF Serving or Seldon • Additional components for storage, workflow Demo: Run TensorFlow Training with Containers Demo: Serving the Model with TF Serving • Options for serving • Wrap model in a web framework (eg – Flask) • Tensorflow Serving • Seldon Demo:

PaaS 层 UI (e.g. dashboard, cli) 用户 CUE schema/模板 “客户端”抽象 标准化的“服务端”抽象 – 应用模型 Open Application Model (OAM) • 通过 OAM spec 定义“以应用为中心”的原语 • 打破“谷仓”! Common Traits Function Deployment K8s Operator Manual Scaler K8s Operators Kubernetes + OAM K8s Plugin HPA Deployment scale-to-0 Function Unified Model Layer Platform Capability Pool 统一的模型层 平台统一“能力池” 模块化的交付系统 - GitOps “应用”配置 Git (as source of truth) Controller 持续交付 KubeVela “The Extensible Application Platform Based on Kubernetes and Open Application Model (OAM)” KubeVela = OAM Kubernetes Runtime + Capability Center + UI (Cli + Dashboard) KubeVela Ø

environment ○ Volume of data ○ Re-keying method ○ Number of key copies ○ Personnel turnover ○ Threat model ○ New and disruptive technologies, e.g., quantum computers Key rotation: compliance PCI DSS v3 {DEK3}KEKv3 Nov 12-Dec 12 Dec 12 - Jan 11 Jan 11 - Feb 10 KEKv1 KEKv2 KEKv3 KMS plugin: threat model and concerns ● KMS server is compromised ● KMS plugin is compromised ● Auth token for KMS - offline In external secret store Kubernetes secrets: summary ● Use encryption based on your threat model, e.g., two layers, like full-disk + application-layer ● Rotate keys regularly to limit the impact

Deployment Function 应用层 能力管理 用户体验层 Kubernetes Open Application Model（OAM） 一个用来构建云原生应用管理平台的标准规范与核心框架 OAM + OAM Platform UI Open Application Model Platform Kubernetes GitOps/持续集成 标准化定义应用组件 标准化配置应用运维能力 alibaba.com path: / service_port: 8001 # 2nd component - componentName: redis Open Application Model Platform 部署 应用配置 （Application Configuration） 面向应用维度配置运维能力与组件 apiVersion: core.oam.dev/v1alpha2

Open Tech Mini Academy @ IBM http://ibm.biz/opentech-ma Kubernetes Resource Model A common resource model can satisfy any deployment requirements § Config Maps § Daemon Sets § Deployments

contributors can get involved in the SIG. Kubernetes is in the process of moving to a new “out of tree” model, this effort spans all the touching points with the underlying infrastructure: compute, storage, have independent feature and patch release cycles, learn how SIG VMware is working to meet this new model on VMware platforms. Agenda 4 What is the VMware SIG Purpose, Projects managed, How to join