Node Operator: Kubernetes Node Management Made Simple 陈俊(Joe), Ant Financial Agenda • Background and Motivation • Introduction of Operators • Node-Operator • Advanced Topic: Teardown Cluster fast and convenient • Add & delete Node at any time • Upgrade Master & Node Components reliably • Canary Rollout • Master & Node Component Versions Management Motivation: Work Order Order Deployment Worker Order • Upgrade Nodes Versions • Upgrade Node 10.10.10.1 • Upgrade docker • Upgrade kubelet • Upgrade Node 10.10.10.2 • Upgrade docker • Upgrade kubelet …. Motivation: Work Order

10-Annotation 11-K8s架构及基本概念 12-Master与Node的通信 13-Node 14-Pod 15-Replica Set 16-Deployment 17-StatefulSet 18-Daemon Set 19-配置最佳实践 20-管理容器的计算资源 21-Kubernetes资源分配 22-将Pod分配到Node 23-容忍与污点 24-Secret 25-Pod优先级和抢占 主机规划 IP 作⽤ 172.20.0.87 ansible-client 172.20.0.88 master,node 172.20.0.89 master,node 172.20.0.90 node 172.20.0.91 node 172.20.0.92 node 准备⼯作 关闭selinux 所有机器都必须关闭selinux，执⾏如下命令即可。 ~]# setenforce /proc/sys/net/bridge/bridge-nf-call-iptables ~]# sysctl -w net.ipv4.ip_forward=1 如果关闭了防⽕墙，则只需执⾏最下⾯三⾏。 在node机器上 ~]# firewall-cmd --permanent --add-port=10250/tcp ~]# firewall-cmd --permanent --add-port=10255/tcp

Cluster 1 Node Rancher Management Server Cluster Customer B Cluster 1 Node Node Control Plane Worker etcd Node Node Node Node Node Node Node All-in-one nodes (cp/etcd/worker) Node Node Node Node Node Node Node Node Node Node Node Node Control Plane Worker etcd MSP Admin Customer B DevOps: End user Customer A DevOps: End user Copyright © SUSE 2021 Namespace/Container as a Service Rancher (cp/etcd/worker) Node Node Node Namespace as a Service Managed Shared Kubernetes Cluster 1 Node Node Node Node 64 GB 16VCPU Worker Master Nodes Node 64 GB 16VCPU Node 64 GB 16VCPU NS: Customer

Contents CIS 1.6 Kubernetes Benchmark - Rancher v2.5.4 with Kubernetes v1.18 Controls 1.1 Etcd Node Configuration Files 1.1.11 Ensure that the etcd data directory permissions are set to 700 or more is not set to AlwaysAllow (Automated) 1.2.8 Ensure that the --authorization-mode argument includes Node (Automated) 1.2.9 Ensure that the --authorization-mode argument includes RBAC (Automated) 1.2.10 Ensure (Automated) 1.4.2 Ensure that the --bind-address argument is set to 127.0.0.1 (Automated) 2 Etcd Node Configuration Files 2.1 Ensure that the --cert-file and --key-file arguments are set as appropriate

identification 4.2. Network 5. Cellar nodes 5.1. Nodes identification 5.2. Testing nodes 5.3. Node Components: listener, producer, handler, consume, and synchronizer 5.4. Synchronizers and sync policy Please note that this behavior is disabled by default as it can have side effects (especially when a node is stopped). Enabling listeners is at your own risk. The nodes list could be discovered (using unicast default Cellar topology. Cellar is installed on all nodes, each node has the same function. It means that you can perform actions on any node, it will be broadcasted to all others nodes. 1.3. Star topology

51 k8s-master2.cof-lee.com 10.99.1.52 k8s-master3.cof-lee.com 10.99.1.53 k8s-node01.cof-lee.com 10.99.1.61 k8s-node02.cof-lee.com 10.99.1.62 规划Pod网络： 10.244.0.0/16 规划Service网络： 10.7.0.0/16 # 99.1.53 k8s-master3.cof-lee.com k8s-master3 10.99.1.61 k8s-node01.cof-lee.com k8s-node01 10.99.1.62 k8s-node02.cof-lee.com k8s-node02 EOF ★k8s初始化时要求系统里有/etc/resolv.conf文件及系统对外通信网口上配置有 默认路由；根据实际情况添加 #加载配置 ⑧防火墙放行端口 TCP: 6443，2379，2380，10250~10252，30000~32767 UDP: 8285，8472 ★最好是允许整个k8s的node网段以及pod网段入站 # firewall-cmd --add-rich-rule='rule family="ipv4" source address="10.99.1.0/24" accept'

identification 4.2. Network 5. Cellar nodes 5.1. Nodes identification 5.2. Testing nodes 5.3. Node Components: listener, producer, handler, consume, and synchronizer 5.4. Synchronizers and sync policy Please note that this behavior is disabled by default as it can have side effects (especially when a node is stopped). Enabling listeners is at your own risk. The nodes list could be discovered (using unicast default Cellar topology. Cellar is installed on every nodes, each node has the same function. It means that you can perform actions on any node, it will be broadcasted to all others nodes. 1.3. Star topology

the European Union and other countries. Node.js ® is an official trademark of Joyent. Red Hat is not formally related to or endorsed by the official Joyent Node.js open source or commercial project. The SELinux 布尔值 5.3.4. 为节点添加内核参数 5.4. 管理每个节点的 POD 数量上限 5.4.1. 配置每个节点的最大 pod 数量 5.5. 使用 NODE TUNING OPERATOR 5.5.1. 访问 Node Tuning Operator 示例规格 5.5.2. 自定义调整规格 5.5.3. 在集群中设置默认配置集 5.5.4. 支持的 TuneD 守护进程插件 5 其他资源 5.7. 使用 NODE HEALTH CHECK OPERATOR 部署节点健康检查 5.7.1. 关于 Node Health Check Operator 5.7.1.1. 了解 Node Health Check Operator 工作流 5.7.1.2. 关于节点健康检查如何防止与机器健康检查冲突 5.7.2. 使用 Web 控制台安装 Node Health Check

Rancher RKE2 Cluster | 59 Configure a Server Node | 59 Configure an Agent Node | 63 Configure Repository Credentials | 66 Prepare a Cluster Node for DPDK | 67 Juniper CN2 Technology Previews controllers manage a distributed set of data planes implemented by a CNI plug-in and vRouter on every node. Integrating a full-fledged vRouter alongside the workloads provides CN2 the flexibility to support such as link and node failures. The Contrail controller reports and logs these events where appropriate and reconfigures the vRouter data plane as necessary. Although any single node can contain only

Controls 1 Master Node Security Configuration 1.1 Master Node Configuration Files 1.2 API Server 1.3 Controller Manager 1.4 Scheduler 2 Etcd Node Configuration 2 Etcd Node Configuration Files 3 Control Plane Configuration 3.2 Logging 4 Worker Node Security Configuration 4.1 Worker Node Configuration Files 4.2 Kubelet 5 Kubernetes Policies 5.1 RBAC and Service Accounts 5.2 Pod Security guide. Controls CIS Benchmark Rancher Self-Assessment Guide - v2.4 5 1 Master Node Security Configuration 1.1 Master Node Configuration Files 1.1.1 Ensure that the API server pod specification file permissions