什么是ServiceAccount

ServiceAccount是Kubernetes中容器用来访问default空间下kubernetes服务的认证机制。我们可以通过kubectl get svc来查看kubernetes服务：

1
2
3
root@fankang:/home/fankang# kubectl get svc kubernetes
NAME         CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
kubernetes   10.100.0.1   <none>        443/TCP   189d

访问kubernetes等同于kube-apiserver的认证端口。当把ServiceAcount的信息传递给容器后，容器中的进程便可以使用ServiceAccount包含的Secret访问kubernetes服务，即10.100.0.1地址。

那么哪些服务需要访问kubernetes服务呢，比如heapster，ingress-controller等，这些都需要从kube-apiserver获取系统资源。

生成ServiceAccount

先来看下如何搭建带ServiceAccount的Kubernetes环境。我的实验环境是虚拟机单机环境，为了方便测试，所有进程都是直接从二进制文件拉起，所以读者需要依据自己的环境修改启动参数。关于key和config机制，请参考另外的相关文章。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
(dockerd -H 0.0.0.0:4243 -H unix:///var/run/docker.sock &>> /var/log/docker.log &)
(etcd --data-dir=/home/fankang/data/etcd150/ --listen-client-urls http://0.0.0.0:4001 --advertise-client-urls http://0.0.0.0:4001 &>> /var/log/etcd.log &)
(启动etcd后稍后启动kubenetes进程)
(kube-apiserver --logtostderr=true --v=0 --etcd-servers=http://localhost:4001 --insecure-bind-address=0.0.0.0 --allow-privileged=false --service-cluster-ip-range=10.100.0.0/16 --admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,ResourceQuota,ServiceAccount --client-ca-file=/etc/kubernetes/security/serverkey/ca.crt --tls-private-key-file=/etc/kubernetes/security/serverkey/server.key --tls-cert-file=/etc/kubernetes/security/serverkey/server.crt &>> /var/log/kube-apiserver.log &)
(kube-controller-manager --logtostderr=true --v=0 --kubeconfig=/root/.kube/config --service_account_private_key_file=/etc/kubernetes/security/serverkey/server.key --root-ca-file=/etc/kubernetes/security/serverkey/ca.crt &>> /var/log/kube-controller-manager.log &)
(kube-scheduler --logtostderr=true --v=0 --kubeconfig=/root/.kube/config &>> /var/log/kube-scheduler.log &)
(kube-proxy --logtostderr=true --v=0 --kubeconfig=/root/.kube/config &>> /var/log/kube-proxy.log &)
(kubelet --logtostderr=true --v=0 --address=0.0.0.0 --allow-privileged=false --pod-infra-container-image=pause:0.8.0 --cpu-cfs-quota=true --kubeconfig=/root/.kube/config --require-kubeconfig=True &>> /var/log/kubelet.log &)

下面我们以default空间为例来查看ServiceAccount。

进程启动完成后，可以通过kubectl get secret查看在default空间(其他空间下类似)下是否有default-token:

1
2
3
root@fankang:/home/fankang# kubectl get secret
NAME                  TYPE                                  DATA      AGE
default-token-zg893   kubernetes.io/service-account-token   3         8d

default-token中包含有ca.crt, namespace, token三个字段信息，且都是base64编码。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
root@fankang:/home/fankang# kubectl get secret default-token-zg893 -o yaml
apiVersion: v1
data:
  ca.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUQ5VENDQXQyZ0F3SUJBZ0lKQUpQbWJ1cm4xNXAzTUEwR0NTcUdTSWIzRFFFQkN3VUFNSUdRTVFzd0NRWUQKVlFRR0V3SkRUakVSTUE4R0ExVUVDQXdJV21obGFtbGhibWN4RVRBUEJnTlZCQWNNQ0VoaGJtZDZhRzkxTVJBdwpEZ1lEVlFRS0RBZG9hV3RrWVhSaE1SQXdEZ1lEVlFRTERBZG9hV3RrWVhSaE1STXdFUVlEVlFRRERBcHJkV0psCmNtNWxkR1Z6TVNJd0lBWUpLb1pJaHZjTkFRa0JGaE5tWVc1cllXNW5RR2hwYTJSaGRHRXVZMjl0TUI0WERURTMKTURZeU9URXlNamt5TUZvWERUSTNNRFl5TnpFeU1qa3lNRm93Z1pBeEN6QUpCZ05WQkFZVEFrTk9NUkV3RHdZRApWUVFJREFoYWFHVnFhV0Z1WnpFUk1BOEdBMVVFQnd3SVNHRnVaM3BvYjNVeEVEQU9CZ05WQkFvTUIyaHBhMlJoCmRHRXhFREFPQmdOVkJBc01CMmhwYTJSaGRHRXhFekFSQmdOVkJBTU1DbXQxWW1WeWJtVjBaWE14SWpBZ0Jna3EKaGtpRzl3MEJDUUVXRTJaaGJtdGhibWRBYUdsclpHRjBZUzVqYjIwd2dnRWlNQTBHQ1NxR1NJYjNEUUVCQVFVQQpBNElCRHdBd2dnRUtBb0lCQVFDNzFrQmxaelp6Q1RhdDVIcm1FaUR5bTdLS1BoRm5nRkx5YzZmSmI0bTVVU0g0CkJWaitiTmYrYzkyd3hRZnNHTy9XNTlXaXV0T1N3YXgvSVZuWlBBRmd4NnEvNGdnR0l0VDdwVFFUbDR2SWpkc1IKMW1ZZmdqd0h5dThIOWxRYlRJOElFT3poMUozUzAvTVUwWUozdTFLOW9YT3JDa2Zha3NRWXVtVVdSbWVGTCtpTgp6azM5UGRPenE4SnFKajNqOWgxK3hIaEQzZC9MYkRqY0xDMWhjbHM4VldCWk9CR1QrVlR0elIwTHZzWEhKOVlhCkJ6YTlKYVJIVEpTeEU2NE1JNHRDR3B2TytOZjZOdjZnTUdkN2pJNnB0TDhabS8yYWs2R2tRRnRUVnNnam5lSFoKdXpOY0hkTnFTVEIycTZDNEZVWWZybzFDUkhaWUVrZWFWVSswSUNSMUFnTUJBQUdqVURCT01CMEdBMVVkRGdRVwpCQlNEMHBjeUMxNkpCSTA4TTNwWlI2WjlZMDNHVFRBZkJnTlZIU01FR0RBV2dCU0QwcGN5QzE2SkJJMDhNM3BaClI2WjlZMDNHVFRBTUJnTlZIUk1FQlRBREFRSC9NQTBHQ1NxR1NJYjNEUUVCQ3dVQUE0SUJBUUFZeXFtRm92ZDkKby9Ka1dzSlY5VXgzdWc4c0VWSHNSNXJTdXpSakJuWUQzakw1MkFFTDdKTjB0YTdwT0h3N3BaQWgyZXpobTJKSwprYnNGbjRpbzRiZ21TN2FLdHgxeGRVbnIvbXovMUhBbVFHS1dMQnFSMDdLUm9zY0JvdkpwMEpyYjlrQVk3Tm9SCnhONGRTNzBDM0ZvNlZHSTBrL3lwL1hnZkZnZU80R24yTnJiT0dDODRiNjBhYS9JWTM2T09uNFpUdFJSS0pZOUMKMDQwNyt1SmV6TThvakdkakRaMGpIN3hoMDJ3dFBWZTlnRk4yblpEVW5YYUF1SGtvbVJmWGFKS0pFbDNhcjZoRgplYjlHLzZVMURBTlkvK3JTRmZ5dFU1WCt6S3RqbjRudGRmMGRJckdtZ2s5d3plUFg3RWZZYWdvbW1MVUd0S1B4CmY4aTQ2VWJ6YUhGeAotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
  namespace: ZGVmYXVsdA==
  token: ZXlKaGJHY2lPaUpTVXpJMU5pSXNJblI1Y0NJNklrcFhWQ0o5LmV5SnBjM01pT2lKcmRXSmxjbTVsZEdWekwzTmxjblpwWTJWaFkyTnZkVzUwSWl3aWEzVmlaWEp1WlhSbGN5NXBieTl6WlhKMmFXTmxZV05qYjNWdWRDOXVZVzFsYzNCaFkyVWlPaUprWldaaGRXeDBJaXdpYTNWaVpYSnVaWFJsY3k1cGJ5OXpaWEoyYVdObFlXTmpiM1Z1ZEM5elpXTnlaWFF1Ym1GdFpTSTZJbVJsWm1GMWJIUXRkRzlyWlc0dGVtYzRPVE1pTENKcmRXSmxjbTVsZEdWekxtbHZMM05sY25acFkyVmhZMk52ZFc1MEwzTmxjblpwWTJVdFlXTmpiM1Z1ZEM1dVlXMWxJam9pWkdWbVlYVnNkQ0lzSW10MVltVnlibVYwWlhNdWFXOHZjMlZ5ZG1salpXRmpZMjkxYm5RdmMyVnlkbWxqWlMxaFkyTnZkVzUwTG5WcFpDSTZJbVptWW1OaE1qazVMVFZqWXprdE1URmxOeTA1TnpZd0xUQTRNREF5TnpjeFpUTTVNeUlzSW5OMVlpSTZJbk41YzNSbGJUcHpaWEoyYVdObFlXTmpiM1Z1ZERwa1pXWmhkV3gwT21SbFptRjFiSFFpZlEuRlN1TF9KMzQzdVl3eUJtSVVjMG02cXZKc2NITUM4YU1MNFJ1eVV4ZmtRcmk1amVlZXhuQzdCT0dqWnJmNURzWWFYa051ajZWb0d6RHk1M0FFdkIxdm5Hb0lhQkNLZXdFY1JyWW9nWmcxTk9pLUNqelFtYS02dE9ySkU2STZkUnJ4Q3dJb2ZsaWlRaTFRdDEwTlRYdVRwaG9IYnZ0X1JXLTBDeV9sdWg1amJQLTZ3RURUTzNEc24xSldfdGZSSktaejEzLWtTQ2RDVV9VS0gzektiaERvLTdTR1R5ZnJOTktDbldGSmd1R3VPWFFJMW15VHVxY0ZDSy1pQjlKNTdCeS1rMzQ1MmtrSTlsVlNRQmZIY3JPbXJaX1JvSWx0RXZxMnBudUFNcExtbWM4OG95RkwzLTFKdXBaV1FMRzQxazJFb0VHVmhYS1ZNOU11bXlubzJweTVn
kind: Secret
metadata:
  annotations:
    kubernetes.io/service-account.name: default
    kubernetes.io/service-account.uid: ffbca299-5cc9-11e7-9760-08002771e393
  creationTimestamp: 2017-06-29T12:55:57Z
  name: default-token-zg893
  namespace: default

其中:

  • ca.crt可以通过cat ca.crt | base64生成;
  • Namespace可以通过echo -n "kube-system" | base64生成;
  • Token随机生成。

然后查看default空间下是否有default ServiceAccount：

1
2
3
root@fankang:/home/fankang# kubectl get sa
NAME      SECRETS   AGE
default   1         8d

此时的default ServiceAccount包含1个Secret，Secret名字就是上面提到的default-token:

1
2
3
4
5
6
7
8
9
10
11
12
root@fankang:/home/fankang# kubectl get sa default -o yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  creationTimestamp: 2017-06-29T12:53:50Z
  name: default
  namespace: default
  resourceVersion: "1243200"
  selfLink: /api/v1/namespaces/default/serviceaccounts/default
  uid: ffbca299-5cc9-11e7-9760-08002771e393
secrets:
- name: default-token-zg893

如果default ServiceAccount中不包含Secret，那么很可能是default-token不存在，请检查kube-controller-manager的–service_accoujnt_private_key_file和–root-ca-file两个参数是否正确，并重新启动kube-controller-manager。

ServiceAccount使用

因为我们在kube-apiserver中的–admision-control中已经开启了ServiceAccount，那么我们就可以直接起容器来使用ServiceAccont。

我们来启动一个ubuntu-ssh的Deployment。Deployment代码如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
apiVersion: extensions/v1beta1
kind: Deployment
metadata: 
  name: ubuntu-ssh
  namespace: default
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: ubuntu-ssh
    spec:
      containers:
      - name: ubuntu-ssh
        image: ubuntu-ssh:v1
        env:
        - name: ROOT_PASSWORD
          value: "123456"
        ports:
        - containerPort: 22
          name: ssh
        resources:
          limits:
            cpu: 100m
            memory: 100Mi
          requests:
            cpu: 100m
            memory: 100Mi

待Pod启动完成后，可以看到容器中已经把default-token挂载到容器的/var/run/secrets/kubernetes.io/servicesaccount目录:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
root@fankang:/home/fankang# kubectl describe pods ubuntu-ssh-589933015-2lhgb
Name:       ubuntu-ssh-589933015-2lhgb
Namespace:  default
Node:       fankang/10.0.2.15
Start Time: Fri, 30 Jun 2017 14:32:40 +0800
Labels:     app=ubuntu-ssh
        pod-template-hash=589933015
Status:     Running
IP:     172.17.0.7
Controllers:    ReplicaSet/ubuntu-ssh-589933015
Containers:
  ubuntu-ssh:
    Container ID:   docker://5eb46b23c2d2237c98c6d6c0b47bdf37db9ede2ba2170a62aae8b3ecdc178574
    Image:      ubuntu-ssh:v1
    Image ID:       docker://sha256:27312dad4272a61739c317b90bfab5bc129e38696df5c606b1041199caf03ea4
    Port:       22/TCP
    Limits:
      cpu:  100m
      memory:   100Mi
    Requests:
      cpu:      100m
      memory:       100Mi
    State:      Running
      Started:      Sat, 08 Jul 2017 18:35:40 +0800
    Last State:     Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Fri, 07 Jul 2017 14:13:58 +0800
      Finished:     Fri, 07 Jul 2017 15:33:17 +0800
    Ready:      True
    Restart Count:  2
    Volume Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token (ro)
    Environment Variables:
      ROOT_PASSWORD:    123456
Conditions:
  Type      Status
  Initialized   True 
  Ready     True 
  PodScheduled  True 
Volumes:
  default-token:
    Type:   Secret (a volume populated by a Secret)
    SecretName: default-token-zg893
QoS Class:  Guaranteed
Tolerations:    <none>
Events:
  FirstSeen LastSeen    Count   From            SubObjectPath           Type        Reason          Message
  --------- --------    -----   ----            -------------           --------    ------          -------
  36m       36m     2   {kubelet fankang}                   Warning     MissingClusterDNS   kubelet does not have ClusterDNS IP configured and cannot create Pod using "ClusterFirst" policy. Falling back to DNSDefault policy.
  36m       36m     1   {kubelet fankang}   spec.containers{ubuntu-ssh} Normal      Pulled          Container image "ubuntu-ssh:v1" already present on machine
  36m       36m     1   {kubelet fankang}   spec.containers{ubuntu-ssh} Normal      Created         Created container with docker id 5eb46b23c2d2; Security:[seccomp=unconfined]
  36m       36m     1   {kubelet fankang}   spec.containers{ubuntu-ssh} Normal      Started         Started container with docker id 5eb46b23c2d2

在容器/var/run/secrets/kubernetes.io/servicesaccount目录下有ca.crt, namespace, token三个文件，这和default-token中的内容是对应的。然后到/var/run/secrets/kubernetes.io/serviceaccount目录下，执行下面语句访问kubernetes服务。

1
curl -H "Authorization: Bearer `cat token`" --cacert ca.crt -XGET https://10.100.0.1:443/api/v1/namespaces/default/pods -v

自挂载使用ServiceAccount

上节介绍了kube-apiserver开启ServiceAccount admission-controller情况下如何使用ServiceAccount。在该情况下，所有创建的Pod都会使用ServiceAccount，从而拥有访问kubernetes服务的能力。这当然不是我们希望看到的，我们希望部分容器可以访问kubernetes服务。

我们可以直接通过挂载ServiceAccount对应的secret来使用ServiceAccount。如下所示：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
apiVersion: extensions/v1beta1
kind: Deployment
metadata: 
  name: ubuntu-ssh
  namespace: default
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: ubuntu-ssh
    spec:
      containers:
      - name: ubuntu-ssh
        image: ubuntu-ssh:v1
        env:
        - name: ROOT_PASSWORD
          value: "123456"
        ports:
        - containerPort: 22
          name: ssh
        resources:
          limits:
            cpu: 100m
            memory: 100Mi
          requests:
            cpu: 100m
            memory: 100Mi
        volumeMounts:
        - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
          name: default-token
          readOnly: true
      volumes:
      - name: default-token
        secret:
          defaultMode: 420
          secretName: default-token-zg893

使用自定义ServiceAccount

一般来说，只要创建完ServiceAccount，kube-controller-manager会自动创建对应的secret，刚创建的ServiceAccount就有访问kubernetes的服务的权限。创建ServiceAccount的YAML文件如下：

1
2
3
4
5
apiVersion: v1
kind: ServiceAccount
metadata:
  name: test-sa
  namespace: default