Highest Level • Container Cluster = “Desired State Management” – Kubernetes Cluster Services (w/API) • Node = Container Host w/agent called “Kubelet” • Application Deployment File = Configuration File be running Worker Node Worker Node Worker Node Kubernetes Master Node (Master & etcd nodes) API K K K App_Y.yaml ContainerImage1 Replicas: 1 ContainerImage2 Replicas: 2 https://youtu.be/PH-2FfFD2PU Creates virtual IP for external access • Interfaces with local iptables • Load-balance interface for Pods • Creates virtual IP for external access • Interfaces with local iptables The Kubernetes Runtime

Provider Background: K8s Secrets • Encryption Keys stored on API Server • Secrets encrypted prior to storage in etcd • Secrets decrypted on API Server prior to use • Encryption keys stored in a remote in the clear in memory ü leak ALL DEKs ü leak ALL secrets ü trust collapse! • DEK decryption interfaces invoked by fake users Motivation: K8s Secrets Protection • Kube-on-Kube [1] ü Components => Host (KMS provider) compromise Ø leak DEKs Ø leak Secrets • Fraudsters calling DEK decryption interfaces TEE-based Kubectl • Address security threats • Client compromise Ø kubeconfig maliciously reused

using multiple interfaces SR-IOV NFV Environments Virtlet Cons limited storage options more configurations VM actions limited by Pod KubeVirt Building a virtualization API for Kubernetes https://github

kubeadm init --kubernetes- version=v1.19.4 \ --apiserver-adver�se- address=10.99.1.51 \ # api server地址 --pod-network-cidr=10.244.0.0/16 \ # pod容器网段 --service-cidr=10.7.0.0/16 \ # service网段，即cluster kubeadm init --kubernetes- version=v1.28.2 \ --apiserver-adver�se- address=10.99.1.51 \ # api server地址 --pod-network-cidr=10.244.0.0/16 \ # pod容器网段 --service-cidr=10.7.0.0/16 \ # service网段，即cluster 高可用集群拓扑图： ★先配置HA高可用的反向代理 本例中vip为10.99.1.54（三台master ip为10.99.1.51~53）使用haproxy做反向代理 frontend k8s_api_tcp_6443 bind *:6443 mode tcp default_backend my_k8s_cluster_6443 backend my_k8s_cluster_6443

01-什么是Kubernetes 02-安装单机版Kubernetes 03-使⽤Kubespray部署⽣产可⽤的Kubernetes集群（1.11.2） 04-K8s组件 05-Kubernetes API 06-理解K8s对象 07-Name 08-Namespace 09-Label和Selector 10-Annotation 11-K8s架构及基本概念 12-Master与Node的通信 允许⽤户使⽤⾃定义信息来装饰资源以⽅便他们的⼯作流程， 并为管理⼯具提供检查点状态的简单⽅法。 此外， Kubernetes control plane 所⽤的API 与开发⼈员和⽤户可⽤的API相同。⽤户可以使⽤ their own API 编写⾃⼰ 的控制器，例如 scheduler ，这些API可由通⽤ command-line tool 定位。 这种 design 使得许多其他系统可以构建在Kubernetes上。 io/article/8136 启⽤Kubernetes Dashboard 执⾏： kubectl proxy 02-安装单机版Kubernetes 8 访问： http://localhost:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/#!/overview? namespace=default

Install API server - Config them correctly - Start them Installation - etcd - SSH - Install etcd - Config them correctly - Start them Installation kops, kubeup.sh, kube-AWS,... AWS, GCP API node1 AWS, GCP API node1 node2 node3 Rollback ??? AWS, GCP API node1 node2 node3 Healing AWS, GCP API node2 node3 Healing AWS, GCP API node2 node3 node1’ Create node Healing AWS, GCP API node2 node3 self-hosted Kubernetes? ● Kubernetes manages own core components ● Core components deployed as native API objects Self-hosted k8s Architecture Why Self-host Kubernetes? ● Operational expertise around app

OpenCost 79. OpenRewrite 80. OrbStack 81. Pixie 82. Tabnine 暂缓 — 采纳 83. Playwright 试验 84. .NET Minimal API 85. Ajv 86. Armeria 87. AWS SAM 88. Dart 89. fast-check 90. Kotlin with Spring 91. Mockery 92. Netflix OpenTelemetry 94. Polars 95. Pushpin 96. Snowpark 评估 97. 基准配置文件 98. GGML 99. GPTCache 100. 语法性别 API 101. htmx 102. Kotlin Kover 103. LangChain 104. LlamaIndex 105. promptfoo 106. Semantic Kernel 107 构文件一样最终被归档和遗忘。 3. 具有可访问性意识的组件测试设计 试验 在软件交付进程中，可访问性要求是 Web 组件测试阶段的一种考察指标。尽管诸如 chai-a11y-axe 的测试框架 插件 API 已提供了基础的可访问性断言，具有可访问性意识的组件测试设计依然能够帮助测试进一步检验屏幕 阅读器和其他辅助技术所需的全量语义元素。 首先，在测试验证元素时，通过 ARIA 角色或者元素的其它语义化属性查找元素，而不采用元素的

targetAverageUtilization: 50 • API Object Oriented Programming Core of API “OO” 1.API objects stores in etcd 2.Control loops (Sync Loop) to reconcile API objects Example kubelet SyncLoop proxy proxy 1 Pod created etcd scheduler api-server Example kubelet SyncLoop kubelet SyncLoop proxy proxy 2 Object added etcd scheduler api-server Example kubelet SyncLoop kubelet SyncLoop a node etcd scheduler api-server Example kubelet SyncLoop kubelet SyncLoop proxy proxy 4.1 Detected bind operation 4.2 Start Pod on this machine etcd scheduler api-server Pattern 1: Controller

Services (Networking, Storage, DNS, Load Balancer, Security) master master api api © 2017 Rancher Labs, Inc. Kubernetes 1.7的扩展特性 • API aggregation(beta) • CustomResourceDefinitions(beta) • Support for extensible flexible way to extend managed resource into a current Kubernetes cluster • Auto-generated API in Kubernetes API server • Customized resource controller to implement your business logic of managed resource and Resource Item my-crontab.yaml © 2017 Rancher Labs, Inc. How Does The Controller Work ETCD API Server Kubernetes Core controllers added creating running stoped deleted Resource Item Resource

Pod 1. 定时执行的批处理任务 2. 定时任务并发策略 * Allow * Forbid * Replace 3. 支持单任务并发控制 一个简单的编排案例 Client API DB API Proxy DB Proxy DB Backup Monitoring Deployment StatefulSet Service CronJob DaemonSet 重新审视这个例子 Client API DB API Proxy DB Proxy DB Backup Monitoring 无状态应用 有状态应用 守护型应用 批处理任务 应用编排架构 应用编排架构 API Gateway APP API Service A APP API Service B APP Cache APP API Service D APP File Kubernetes Cluster Application Registry Application Manager Application A Application … API Gateway API Service A API Service B 应用编排架构 Application Registry - Helm Registry Helm Chart Helm Registry 1