6-K8S核心技术Controller

发表于 更新于 分类于

k8s.jpg

概念

k8s通常不会直接创建pod，而是通过controller来管理pod，controller中定义了pod的部署特性，为对应不同场景，k8s提供了多种controller。

  1. ReplicationController(RC) ：上一代无状态pod应用控制器，不建议使用，建议使用Deployment、ReplicaSet来取代

  2. ReplicaSet(RS) ：新一代的ReplicationController实现了pod的多副本管理，使用Deployment时会自动创建ReplicaSet，也就是说Deployment是通过ReplicaSet来管理pod的多个副本，通常不使用ReplicaSet，而是通过Deployment去使用ReplicaSet(ReplicaSet不支持滚动更新，但是Deployment支持)

  3. Deployment(deploy) ：用于管理无状态应用，是构建在ReplicaSet之上的，更为高级的控制器

  4. StatefuleSet(sts) ：用于管理有状态应用，如数据库服务程序，与Deployment不同的是StatefulSet会为每一个pod创建一个独有的持久性标识，并确保pod间的顺序。有序收缩、有序删除

  5. DeamonSet(ds) ：用于确保每一个节点都运行了某pod的一个副本，包括后来新增的节点，节点移除则导致pod回收。

  6. job ：用于管理运行完成后即可终止的应用。

  7. cronjob ：用于管理定时反复执行的任务

Deployment（deploy）

创建deployment

更多参数查看官方api文档
编辑

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
apiVersion: apps/v1   # Deployment所在的api
kind: Deployment      # 资源类型
metadata:
  name: nginx-deployment  # deployment名
  labels:                 # deployment标签
    app: nginx
spec:                       #
  replicas: 3               # pod资源副本数
  selector:                 # 管理pod的标签
    matchLabels:            # 相当于使用matchExpressions {key: app, operator: In, values: nginx}
      app: nginx
  template:                 # 定义要创建的pod
    metadata:               # pod的元数据
      labels:               # pod的标签
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80   #指定要暴露的端口

使用以上yaml创建nginx-deployment.yaml文件 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
~]# kubectl apply -f nginx-deployment.yaml   # 使用nginx-deployment.yaml创建deployment

~]# kubectl get deploy
NAME               READY        UP-TO-DATE   AVAILABLE   AGE
nginx-deployment   3/3          3            3           15m

~]# kubectl rollout status deployment/nginx-deployment   #查看nginx-deployment deployment发布状态
deployment "nginx-deployment" successfully rolled out

~]# kubectl get pod,rs -o wide --show-labels  #查看由deployment创建的pod、replicaSet
NAME                                    READY   STATUS    RESTARTS   AGE   IP            NODE         NOMINATED NODE   READINESS GATES   LABELS
pod/nginx-deployment-7fd6966748-bhlxj   1/1     Running   0          29m   10.244.2.8    k8s-node02   <none>           <none>            app=nginx,pod-template-hash=7fd6966748
pod/nginx-deployment-7fd6966748-m85w9   1/1     Running   0          29m   10.244.1.11   k8s-node01   <none>           <none>            app=nginx,pod-template-hash=7fd6966748
pod/nginx-deployment-7fd6966748-sx54r   1/1     Running   0          29m   10.244.2.9    k8s-node02   <none>           <none>            app=nginx,pod-template-hash=7fd6966748

NAME                                                DESIRED   CURRENT   READY   AGE   CONTAINERS   IMAGES         SELECTOR                                 LABELS
replicaset.extensions/nginx-deployment-7fd6966748   3         3         3       29m   nginx        nginx:1.14.2   app=nginx,pod-template-hash=7fd6966748   app=nginx,pod-template-hash=7fd6966748

pod-template-hash标签是由spec.template（PodTemplate）生成的hash值，这个标签添加到由deployment生成的pod、ReplicaSet中

更新Deployment

1
2
3
4
5
6
7
8
9
10
11
~]# kubectl set image deployment/nginx-deployment nginx=nginx:1.16.1   #将nginx版本由创建时的1.14.2版本改为1.16.1
deployment.extensions/nginx-deployment image updated

~]# kubectl edit deployment/nginx-deployment  #使用编辑器直接修改nginx的版本

~]# kubectl rollout status deployment/nginx-deployment   #查看nginx-deployment deployment发布状态

~]# kubectl get rs         # 可以发现新的rs扩容到3个，并将旧rs缩容到0个
NAME                          DESIRED   CURRENT   READY   AGE
nginx-deployment-6f9d665859   3         3         3       8h
nginx-deployment-7fd6966748   0         0         0       9h

deployment更新的过程类似于，先创建一个新的rs，等待就绪后，将旧rs缩容到2，也就是创建一个新的，缩容一个旧的，直到新的rs扩容为3，旧的rs缩容为0。

Rollover

Deployment每次更新都会创建新的rs、pod，当Deployment正在更新时再次被更新，正在扩容的rs将被当做为旧rs，然后在进行滚动更新。
例如：当Deployment正在创建5个nginx:1.14.2的副本，在创建了3个的时候将nginx改为nginx:1.16.1，这个时候刚创建的3个将被杀死，并开始创建新的副本，不会等待nginx:1.14.2创建完成后在进行更新

Deployment标签不可变

在 API 版本 apps/v1 中，Deployment 标签选择算符在创建后是不可变的。

回滚Deployment

Deployment的所有上线记录都会保留在系统中，以便随时回滚。 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
~]# kubectl rollout history deployment/nginx-deployment  # 查看deployment/nginx-deployment历史本本
deployment.extensions/nginx-deployment
REVISION  CHANGE-CAUSE
3         <none>
4         <none>
# 可以看到有两个历史版本，但是change-cause是空，CHANGE-CAUSE 的内容是从 Deployment 的 kubernetes.io/change-cause 注解复制过来的。

~]# kubectl annotate deployment/nginx-deployment kubernetes.io/change-cause="image updated to 1.16.1"   
#当前nginx版本为1.16.1，为change-cause设定
deployment.extensions/nginx-deployment annotated

~]# kubectl rollout history deployment/nginx-deployment
deployment.extensions/nginx-deployment
REVISION  CHANGE-CAUSE
3         <none>
4         image updated to 1.16.1

~]# kubectl rollout history deployment/nginx-deployment --revision=4  #查看对应版本的详细记录
deployment.extensions/nginx-deployment with revision #4
Pod Template:
  Labels:       app=nginx
        pod-template-hash=6f9d665859
  Annotations:  kubernetes.io/change-cause: image updated to 1.16.1
  Containers:
   nginx:
    Image:      nginx:1.16.1
    Port:       80/TCP
    Host Port:  0/TCP
    Environment:        <none>
    Mounts:     <none>
  Volumes:      <none>

~]# kubectl rollout undo deployment/nginx-deployment --to-revision=3  #回滚到版本3
deployment.extensions/nginx-deployment rolled back

~]# kubectl describe deploy nginx-deployment  #查看当前nginx-deployment描述信息
Name:                   nginx-deployment
Namespace:              default
CreationTimestamp:      Sat, 02 Jul 2022 21:56:47 +0800
Labels:                 app=nginx
Annotations:            deployment.kubernetes.io/revision: 5
                        kubectl.kubernetes.io/last-applied-configuration:
                          {"apiVersion":"apps/v1","kind":"Deployment","metadata":{"annotations":{},"labels":{"app":"nginx"},"name":"nginx-deployment","namespace":"d...
Selector:               app=nginx
Replicas:               3 desired | 3 updated | 3 total | 3 available | 0 unavailable
StrategyType:           RollingUpdate
MinReadySeconds:        0
RollingUpdateStrategy:  25% max unavailable, 25% max surge
Pod Template:
  Labels:  app=nginx
  Containers:
   nginx:
    Image:        nginx:1.14.2
    Port:         80/TCP
    Host Port:    0/TCP
    Environment:  <none>
    Mounts:       <none>
  Volumes:        <none>
Conditions:
  Type           Status  Reason
  ----           ------  ------
  Available      True    MinimumReplicasAvailable
  Progressing    True    NewReplicaSetAvailable
OldReplicaSets:  <none>
NewReplicaSet:   nginx-deployment-7fd6966748 (3/3 replicas created)
Events:
  Type    Reason             Age                From                   Message
  ----    ------             ----               ----                   -------
  Normal  ScalingReplicaSet  66s (x2 over 11h)  deployment-controller  Scaled up replica set nginx-deployment-7fd6966748 to 1
  Normal  ScalingReplicaSet  63s (x2 over 11h)  deployment-controller  Scaled down replica set nginx-deployment-6f9d665859 to 2
  Normal  ScalingReplicaSet  63s (x2 over 11h)  deployment-controller  Scaled up replica set nginx-deployment-7fd6966748 to 2
  Normal  ScalingReplicaSet  61s (x2 over 12h)  deployment-controller  Scaled up replica set nginx-deployment-7fd6966748 to 3
  Normal  ScalingReplicaSet  61s                deployment-controller  Scaled down replica set nginx-deployment-6f9d665859 to 1
  Normal  ScalingReplicaSet  59s                deployment-controller  Scaled down replica set nginx-deployment-6f9d665859 to 0

弹性伸缩

1
2
~]# kubectl scale deployment/nginx-deployment --replicas=2  #将deployment的副本数调整为2
deployment.extensions/nginx-deployment scaled

暂停与恢复上线

在deployment上线过程中可以将此过程进行暂停以及恢复，可以在暂停期间进行修改，暂停之前的状态将继续发挥作用，新的更新在暂停期间不会发挥任何作用。暂停期间不能进行回滚 

1
2
3
4
5
~]# kubectl rollout pause deployment/nginx-deployment  #暂停
deployment.extensions/nginx-deployment paused

~]# kubectl rollout resume  deployment/nginx-deployment  #恢复
deployment.extensions/nginx-deployment resumed

deployment状态

查看当前状态

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
~]# kubectl describe deploy nginx-deployment
Name:                   nginx-deployment
Namespace:              default
CreationTimestamp:      Sat, 02 Jul 2022 21:56:47 +0800
Labels:                 app=nginx
Annotations:            deployment.kubernetes.io/revision: 5
                        kubectl.kubernetes.io/last-applied-configuration:
                          {"apiVersion":"apps/v1","kind":"Deployment","metadata":{"annotations":{},"labels":{"app":"nginx"},"name":"nginx-deployment","namespace":"d...
Selector:               app=nginx
Replicas:               2 desired | 2 updated | 2 total | 2 available | 0 unavailable
StrategyType:           RollingUpdate
MinReadySeconds:        0
RollingUpdateStrategy:  25% max unavailable, 25% max surge
Pod Template:
  Labels:  app=nginx
  Containers:
   nginx:
    Image:        nginx:1.14.2
    Port:         80/TCP
    Host Port:    0/TCP
    Environment:  <none>
    Mounts:       <none>
  Volumes:        <none>
Conditions:
  Type           Status  Reason
  ----           ------  ------
  Available      True    MinimumReplicasAvailable
  Progressing    True    NewReplicaSetAvailable
OldReplicaSets:  <none>
NewReplicaSet:   nginx-deployment-7fd6966748 (2/2 replicas created)
Events:
  Type    Reason             Age                From                   Message
  ----    ------             ----               ----                   -------
  Normal  ScalingReplicaSet  50m (x2 over 12h)  deployment-controller  Scaled up replica set nginx-deployment-7fd6966748 to 1
  Normal  ScalingReplicaSet  50m (x2 over 12h)  deployment-controller  Scaled down replica set nginx-deployment-6f9d665859 to 2
  Normal  ScalingReplicaSet  50m (x2 over 12h)  deployment-controller  Scaled up replica set nginx-deployment-7fd6966748 to 2
  Normal  ScalingReplicaSet  50m (x2 over 13h)  deployment-controller  Scaled up replica set nginx-deployment-7fd6966748 to 3
  Normal  ScalingReplicaSet  50m                deployment-controller  Scaled down replica set nginx-deployment-6f9d665859 to 1
  Normal  ScalingReplicaSet  50m                deployment-controller  Scaled down replica set nginx-deployment-6f9d665859 to 0
  Normal  ScalingReplicaSet  46m (x3 over 12h)  deployment-controller  Scaled down replica set nginx-deployment-7fd6966748 to 2

deployment有三种运行状态， Progressing（进行中）、 Complete（已完成）、 Failed（失败），该状态会被添加到conditions中

Progressing（进行中）

  • type: Progressing
  • status: “True”
  • reason: NewReplicaSetCreated | reason: FoundNewReplicaSet | reason: ReplicaSetUpdated

Complete（已完成）

  • type: Progressing
  • status: True
  • reason: NewReplicaSetAvailable

Progressing 会一直持续为true，即使副本的可用状态有变化，Progressing状态也不会发生变化

Failed（失败）

检测该状态的方法是在deployment的资源清单中加入截止时间参数.spec.progressDeadlineSeconds。超过该参数设定的时间范围将会把deployment的状态发送到conditions中

  • Type=Progressing
  • Status=False
  • Reason=ProgressDeadlineExceeded

deployment yaml参数介绍

相关参数查看官方文档deployment API、公共定义ObjectMeta、pod API 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  annotations:
    deployment.kubernetes.io/revision: "5"   #当前deployment版本序号
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"apps/v1","kind":"Deployment","metadata":{"annotations":{},"labels":{"app":"nginx"},"name":"nginx-deployment","namespace":"default"},"spec":{"replicas":3,"selector":{"matchLabels":
{"app":"nginx"}},"template":{"metadata":{"labels":{"app":"nginx"}},"spec":{"containers":[{"image":"nginx:1.14.2","name":"nginx","ports":[{"containerPort":80}]}]}}}}
  creationTimestamp: "2022-07-02T13:56:47Z"  
  generation: 9
  labels:
    app: nginx
  name: nginx-deployment
  namespace: default
  resourceVersion: "155887"
  selfLink: /apis/extensions/v1beta1/namespaces/default/deployments/nginx-deployment
  uid: 9d3bea0f-dfd8-4787-9448-b2e91e46e4a8
spec:
  progressDeadlineSeconds: 600   #进行状态截止时间
  replicas: 2                    #副本数
  revisionHistoryLimit: 10       #历史版本保存数量
  selector:
    matchLabels:
      app: nginx
  strategy:                     #pod更新策略，有rollingUpdate与Recreate（创建新pod前会将所有pod先杀死）两种
    rollingUpdate:
      maxSurge: 25%             #可以超出预期pod量的大小
      maxUnavailable: 25%       #最大不可用百分比
    type: RollingUpdate
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: nginx
    spec:
      containers:
      - image: nginx:1.14.2
        imagePullPolicy: IfNotPresent  #镜像下载策略  IfNotPresent：如果本地有就不检查，如果没有就拉取
        name: nginx
        ports:
        - containerPort: 80
          protocol: TCP
        resources: {}    #容器所需的计算资源
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      restartPolicy: Always    #重启策略
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30
status:
  availableReplicas: 2
  conditions:
  - lastTransitionTime: "2022-07-02T13:58:44Z"
    lastUpdateTime: "2022-07-02T13:58:44Z"
    message: Deployment has minimum availability.
    reason: MinimumReplicasAvailable
    status: "True"
    type: Available
  - lastTransitionTime: "2022-07-03T03:31:18Z"
    lastUpdateTime: "2022-07-03T03:31:18Z"
    message: ReplicaSet "nginx-deployment-7fd6966748" has successfully progressed.
    reason: NewReplicaSetAvailable
    status: "True"
    type: Progressing
  observedGeneration: 9
  readyReplicas: 2
  replicas: 2
  updatedReplicas: 2