Kubernetes VM Solutions for Multi-Tenant Applications Guangxu Li, Senior Software Engineer, ZTE li.guangxu@zte.com.cn Container and VM Ecosystem Kubernetes Docker Swarm Marathon Nomad Container Container OpenStack Others Why We Run VM on Kubernetes? • Traditional Applications • No linux based Applications • Functions provided by host kernel are not satisfied • OpenStack is too complex • Unified

spec: workload: apiVersion: apps/v1 kind: Deployment spec: template: spec: containers: - name: web image: 'php:latest' env: - name: OAM_TEXTURE value: texture.jpg ports: - containerPort: 8001 Traits/Scopes Scale: - 10,000 nodes/cluster - 100,000 apps/cluster - 1,000,000 containers/cluster Applications: - 100,000 deploys/day - 500~1000 replicas/app Function Job Container Virtual Machine Gateway

everything needed to run your application • Build once run anywhere • Starts in seconds: Great for scalability • Images are stored in a centralized place (Docker Hub, Azure Container Registry, gcr, ECR etc • Container orchestration • Automating deployment, scaling, and management of containerized applications • Declarative • Can be a mix of GPU or CPU nodes • Massive Scale • OpenAI dedicates up to Containers Demo: Serving the Model with TF Serving • Options for serving • Wrap model in a web framework (eg – Flask) • Tensorflow Serving • Seldon Demo: Run TensorFlow Training with Kubeflow

server ops – No provisioning, updating, and managing server infrastructure. – Flexible Scalability • No compute cost when idle Benefits Serverless landscape defined in CNCF Kubernetes StatefulSet/DaemonSet • StatefulSet is the workload API object used to manage stateful applications. Manages the deployment and scaling of a set of pods, and provides guarantees about the ordering a logical set of Pods and a policy by which to access them. • Service provides a way for applications to communicate with each other on K8s platform • In OpenWhisk, we deploy all the dependencies

Developer • Hello World! • Containerize Applications • Micro Services Applications • Scalable Kubernetes Applications • Scalable Infrastructure for Applications Application Operating System Physical the App From the OS 2 technologies with 2 different objectives Allows you to run multiple applications on the same OS Allows you to run multiple OS on the same hardware Application Operating System to desired • Policy-based workload scheduler • Topology aware • Assists with availability, performance and capacity • Affinity/Anti-Affinity Capable The Kubernetes Worker Node Basic Components Master

VMware SIG Deep Dive into Kubernetes Scheduling Performance and high availability options for vSphere Steve Wong, Hui Luo VMware Cloud Native Applications Business Unit November 12, 2018 2 Open Source (sites, affinity groups, NUMA, etc.). ​This session will explain the options to gain better performance, resource optimization and availability through tuning of vSphere, and Kubernetes configuration the thread runs on, but can potentially come from other nodes with broad performance implications. Unpredictable performance? Swapping? This basically comes down to a choice of whether you would rather

Balancing loads Rolling updates Monitoring resources Accessing and ingesting logs Debugging applications Providing authentication and authorization 这提供了PaaS的简单性，并具有IaaS的灵活性，并促进了跨基础架构提供商的可移植性。 Kubernetes是⼀个怎样的平台？ Kubernetes： 不限制⽀持的应⽤类型。不规定应⽤框架（例如 Wildfly ），不限制⽀持的语⾔运⾏时（例如Java，Python， Ruby），不局限于 12-factor applications ，也不区分应⽤程序和服务 。 Kubernetes旨在⽀持各种各样的⼯作负 载，包括⽆状态、有状态以及数据处理⼯作负载。 如果应⽤程序可在容器中运⾏，那么它应该能够很好地在 Kubernetes上运⾏。 #!/overview? namespace=default 参考： https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/ 02-安装单机版Kubernetes 9 使⽤Kubespray部署⽣产可⽤的Kubernetes集群 （1.11.2） 前提：科学上⽹，或⾃⾏将gcr.io的镜像转成其他镜像仓库的镜像。

IEEE ISPA 2018 (16th IEEE International Symposium on Parallel and Distributed Processing with Applications) 能力扩展：GPU支持 资源-访问代价树 四类通信方式分类中，通信开销最大的是SOC，其次是PXB，再次是PHB，PIX通信方式 的GPU之间的通信开销最小。 拓扑节点中存储3个信息： 为任务分配相应的硬件 资源及配置容器运行时环境 transparent. GaiaGPU不应修改Kubernetes代码或容器镜像以共享GPU。使用共享GPU执行应用程序应该就像 在物理GPU上执行一样。 Performance. GaiaGPU应当保证vGPU的性能与原生GPU性能相近。 Isolation. GaiaGPU可以有效的分配和回收每个容器使用的GPU资源并实现不同容器间的资源隔离。 Native_time IEEE ISPA 2018 (16th IEEE International Symposium on Parallel and Distributed Processing with Applications) 生态 Next Tencent Kubernetes Engine

DevOps的支持 • 常用DevOps工具的支持，快速融入现有开发体系 Console Restful API K8S-1 K8S-2 XPlatform Push Upload Web Hook 15 2017 Lenovo Internal. All rights reserved. 容器云设计– 存储方案 • 存储方案选择，主要给服务组件提供服务，如MySQL，Redis等 OS CPU/Disk/Memory Kubernetes Docker Applications 20 2017 Lenovo Internal. All rights reserved. 监控与日志 • 日志平台：多种采集手段并行 OS Kubernetes Docker Applications K8S DaemonSet logspout K8S DaemonSet 滚动更新：实时查看更新进度 25 2017 Lenovo Internal. All rights reserved. Showcase • 容器访问：使用exec的api，通过websocket提供web终端 26 2017 Lenovo Internal. All rights reserved. Showcase • 日志查看：实时查看应用日志 27 2017 Lenovo Internal

service stability Ø Quota management to optimize resource orchestration efficiency Ø High performance and comprehensive autoscaling What is TKEx Ø Based on TKE (Tencent Kubernetes Engine) & EKS (Tencent deployment of online and offline services. • Support Service Mesh. • Large-scale and high-performance autoscaling capabilities. • Multi-tenant and quota management. • etc. TKEx Architecture EKS Kernel Kubelet Dockerd self-agent self-agent Patch Node Condition �������������������������� TKEx Web Job Queue Manager Online & Offline Task OfflineTask DynamicQuota Rebalance Worker TKEx-API