文章
文章用户文档快讯圈子网址导航

Kubernetes集群部署

释放双眼，带上耳机，听听看~！

Kubernetes集群部署

1.官方提供的三种部署方式
2.Kubernetes平台环境规划
3.自签SSL证书
4.Etcd数据库群集部署
5.Node安装Docker
6.Flannel容器集群网络部署
7.部署Master组件
8.部署Node组件
9.部署一个测试示例
10.部署Web UI（Dashboard）
11.部署集群内部DNS解析服务（CoreDNS）

官方提供的三种部署方式：

minikube：

Minikube是一个工具，可以在本地快速运行单点的Kubernetes，仅用于尝试Kubernetes或日常开发的用户使用
部署地址：https://kubernetes.io/docs/setup/minikube/

kubeadm：

Kubeadm也是一个工具，提供kubeadm init和kubeadm join，用于快速部署Kubernetes集群
部署地址：https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm/

二进制包：

推荐，从官方下载发行版的二进制包，手动部署每个组件包，组成Kubernetes集群
下载地址：https://github.com/kubernetes/kubernetes/releases

要解决服务发现的问题，需要下面三大支柱，缺一不可

1.一个强一致性，高可用的服务存储目录

基于Ralf算法的etcd天生就是这样一个强一致性，高可用的服务存储目录

2.一秒注册服务和健康服务健康状况的机制

用户可以在etcdz中注册服务，并且对注册的服务配置key TTL，定时保持服务的心跳以达到监控健康状态的效果

3.一种查找和连接服务的机制

通过在etcd指定的主题下注册的服务业能在对应的主题下查到，为了确保连接，我们可以在每个服务机器上都部署一个proxy模式的etcd，这样就可以确保访问etcd集群的服务都能够互相连接

Demo：二进制部署多节点，单etcd群集

Kubernetes集群部署

环境准备：

相关软件包及文档：

链接：https://pan.baidu.com/s/1l4vVCkZ03la-VpIFXSz1dA
提取码：rg99

Mester：7-3：192.168.18.128 kube-apiserver kube-controller-manager kube-scheduler etcd

Node1：7-4：192.168.18.148 kubelet kube-proxy docekr flannel etcd

Node2：7-5：192.168.18.145 kubelet kube-proxy docekr flannel etcd

Mester7-3：



1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
1[root@master ~]# systemctl stop firewalld.service

2[root@master ~]# setenforce 0

3[root@master ~]# mkdir k8s

4[root@master ~]# cd k8s/

5[root@master k8s]# mkdir etcd-cert

6[root@master k8s]# mv etcd-cert.sh etcd-cert

7[root@master k8s]# ls

8etcd-cert  etcd.sh

9[root@master k8s]# vim cfssl.sh

10curl -L https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 -o /usr/local/bin/cfssl

11curl -L https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 -o /usr/local/bin/cfssljson

12curl -L https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64 -o /usr/local/bin/cfssl-certinfo

13chmod +x /usr/local/bin/cfssl /usr/local/bin/cfssljson /usr/local/bin/cfssl-certinfo

14[root@master k8s]# bash cfssl.sh

15[root@master k8s]# ls /usr/local/bin/

16cfssl  cfssl-certinfo  cfssljson

17

18`定义CA证书`

19cat > ca-config.json <<EOF

20{

21  "signing":{

22    "default":{

23      "expiry":"87600h"

24    },

25    "profiles":{

26      "www":{

27        "expiry":"87600h",

28        "usages":[

29          "signing",

30          "key encipherment",

31          "server auth",

32          "client auth"

33        ]

34      }

35    }

36  }

37}

38EOF

39

40`实证书签名`

41cat > ca-csr.json <<EOF

42{

43    "CN":"etcd CA",

44    "key":{

45        "algo":"rsa",

46        "size":2048

47    },

48    "names":[

49        {

50            "C":"CN",

51            "L":"Nanjing",

52            "ST":"Nanjing"

53        }

54    ]

55}

56EOF

57

58`生产证书，生成ca-key.pem  ca.pem`

59[root@master k8s]# cd etcd-cert/

60[root@master etcd-cert]# cfssl gencert -initca ca-csr.json | cfssljson -bare ca -

612020/01/15 11:26:22 [INFO] generating a new CA key and certificate from CSR

622020/01/15 11:26:22 [INFO] generate received request

632020/01/15 11:26:22 [INFO] received CSR

642020/01/15 11:26:22 [INFO] generating key: rsa-2048

652020/01/15 11:26:23 [INFO] encoded CSR

662020/01/15 11:26:23 [INFO] signed certificate with serial number 58994014244974115135502281772101176509863440005

67

68`指定etcd三个节点之间的通信验证`

69cat > server-csr.json <<EOF

70{

71    "CN": "etcd",

72    "hosts": [

73    "192.168.18.128",

74    "192.168.18.148",

75    "192.168.18.145"

76    ],

77    "key": {

78        "algo": "rsa",

79        "size": 2048

80    },

81    "names": [

82        {

83            "C": "CN",

84            "L": "NanJing",

85            "ST": "NanJing"

86        }

87    ]

88}

89EOF

90

91`生成ETCD证书 server-key.pem   server.pem`

92[root@master etcd-cert]# cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=www server-csr.json | cfssljson -bare server

932020/01/15 11:28:07 [INFO] generate received request

942020/01/15 11:28:07 [INFO] received CSR

952020/01/15 11:28:07 [INFO] generating key: rsa-2048

962020/01/15 11:28:07 [INFO] encoded CSR

972020/01/15 11:28:07 [INFO] signed certificate with serial number 153451631889598523484764759860297996765909979890

982020/01/15 11:28:07 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for

99websites. For more information see the Baseline Requirements for the Issuance and Management

100of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);

101specifically, section 10.2.3 ("Information Requirements").

102

103

上传以下三个压缩包进行解压：

Kubernetes集群部署



1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
1[root@master etcd-cert]# ls

2ca-config.json  etcd-cert.sh                          server-csr.json

3ca.csr          etcd-v3.3.10-linux-amd64.tar.gz       server-key.pem

4ca-csr.json     flannel-v0.10.0-linux-amd64.tar.gz    server.pem

5ca-key.pem      kubernetes-server-linux-amd64.tar.gz

6ca.pem          server.csr

7[root@master etcd-cert]# mv *.tar.gz ../

8[root@master etcd-cert]# cd ../

9[root@master k8s]# ls

10cfssl.sh   etcd.sh                          flannel-v0.10.0-linux-amd64.tar.gz

11etcd-cert  etcd-v3.3.10-linux-amd64.tar.gz  kubernetes-server-linux-amd64.tar.gz

12[root@master k8s]# tar zxvf etcd-v3.3.10-linux-amd64.tar.gz

13[root@master k8s]# ls etcd-v3.3.10-linux-amd64

14Documentation  etcd  etcdctl  README-etcdctl.md  README.md  READMEv2-etcdctl.md

15[root@master k8s]# mkdir /opt/etcd/{cfg,bin,ssl} -p

16[root@master k8s]# mv etcd-v3.3.10-linux-amd64/etcd etcd-v3.3.10-linux-amd64/etcdctl /opt/etcd/bin/

17

18`证书拷贝`

19[root@master k8s]# cp etcd-cert/*.pem /opt/etcd/ssl/

20

21`进入卡住状态等待其他节点加入`

22[root@master k8s]# bash etcd.sh etcd01 192.168.18.128 etcd02=https://192.168.18.148:2380,etcd03=https://192.168.18.145:2380

23Created symlink from /etc/systemd/system/multi-user.target.wants/etcd.service to /usr/lib/systemd/system/etcd.service.

24

25

此时新打开一个7-3的远程连接终端：



1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
1[root@master ~]# ps -ef | grep etcd

2root       3479   1780  0 11:48 pts/0    00:00:00 bash etcd.sh etcd01 192.168.18.128 etcd02=https://192.168.195.148:2380,etcd03=https://192.168.195.145:2380

3root       3530   3479  0 11:48 pts/0    00:00:00 systemctl restart etcd

4root       3540      1  1 11:48 ?        00:00:00 /opt/etcd/bin/etcd 

5--name=etcd01 --data-dir=/var/lib/etcd/default.etcd 

6--listen-peer-urls=https://192.168.18.128:2380 

7--listen-client-urls=https://192.168.18.128:2379,http://127.0.0.1:2379 

8--advertise-client-urls=https://192.168.18.128:2379 

9--initial-advertise-peer-urls=https://192.168.18.128:2380 

10--initial-cluster=etcd01=https://192.168.18.128:2380,etcd02=https://192.168.195.148:2380,etcd03=https://192.168.195.145:2380 

11--initial-cluster-token=etcd-cluster 

12--initial-cluster-state=new 

13--cert-file=/opt/etcd/ssl/server.pem 

14--key-file=/opt/etcd/ssl/server-key.pem 

15--peer-cert-file=/opt/etcd/ssl/server.pem 

16--peer-key-file=/opt/etcd/ssl/server-key.pem 

17--trusted-ca-file=/opt/etcd/ssl/ca.pem 

18--peer-trusted-ca-file=/opt/etcd/ssl/ca.pem

19root       3623   3562  0 11:49 pts/1    00:00:00 grep --color=auto etcd

20

21





1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
1`拷贝证书去其他节点`

2[root@master k8s]# scp -r /opt/etcd/ root@192.168.18.148:/opt/

3The authenticity of host '192.168.18.148 (192.168.18.148)' can't be established.

4ECDSA key fingerprint is SHA256:mTT+FEtzAu4X3D5srZlz93S3gye8MzbqVZFDzfJd4Gk.

5ECDSA key fingerprint is MD5:fa:5a:88:23:49:60:9b:b8:7e:4b:14:4b:3f:cd:96:a0.

6Are you sure you want to continue connecting (yes/no)? yes

7Warning: Permanently added '192.168.18.148' (ECDSA) to the list of known hosts.

8root@192.168.18.148's password:

9etcd                                                       100%  518   426.8KB/s   00:00

10etcd                                                       100%   18MB 105.0MB/s   00:00

11etcdctl                                                    100%   15MB 108.2MB/s   00:00

12ca-key.pem                                                 100% 1679     1.4MB/s   00:00

13ca.pem                                                     100% 1265   396.1KB/s   00:00

14server-key.pem                                             100% 1675     1.0MB/s   00:00

15server.pem                                                 100% 1338   525.6KB/s   00:00

16[root@master k8s]# scp -r /opt/etcd/ root@192.168.18.145:/opt/

17The authenticity of host '192.168.18.145 (192.168.18.145)' can't be established.

18ECDSA key fingerprint is SHA256:mTT+FEtzAu4X3D5srZlz93S3gye8MzbqVZFDzfJd4Gk.

19ECDSA key fingerprint is MD5:fa:5a:88:23:49:60:9b:b8:7e:4b:14:4b:3f:cd:96:a0.

20Are you sure you want to continue connecting (yes/no)? yes

21Warning: Permanently added '192.168.18.145' (ECDSA) to the list of known hosts.

22root@192.168.18.145's password:

23etcd                                                       100%  518   816.5KB/s   00:00

24etcd                                                       100%   18MB  87.4MB/s   00:00

25etcdctl                                                    100%   15MB 108.6MB/s   00:00

26ca-key.pem                                                 100% 1679     1.3MB/s   00:00

27ca.pem                                                     100% 1265   411.8KB/s   00:00

28server-key.pem                                             100% 1675     1.4MB/s   00:00

29server.pem                                                 100% 1338   639.5KB/s   00:00

30

31`启动脚本拷贝其他节点`

32[root@master k8s]# scp /usr/lib/systemd/system/etcd.service root@192.168.18.148:/usr/lib/systemd/system/

33root@192.168.18.148's password:

34etcd.service                                               100%  923   283.4KB/s   00:00

35[root@master k8s]# scp /usr/lib/systemd/system/etcd.service root@192.168.18.145:/usr/lib/systemd/system/

36root@192.168.18.145's password:

37etcd.service                                               100%  923   347.7KB/s   00:00

38

39

Node1：7-4



1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
1`修改`

2[root@node1 ~]# systemctl stop firewalld.service

3[root@node1 ~]# setenforce 0

4[root@node1 ~]# vim /opt/etcd/cfg/etcd

5#[Member]

6ETCD_NAME="etcd02"

7ETCD_DATA_DIR="/var/lib/etcd/default.etcd"

8ETCD_LISTEN_PEER_URLS="https://192.168.18.148:2380"

9ETCD_LISTEN_CLIENT_URLS="https://192.168.18.148:2379"

10

11#[Clustering]

12ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.18.148:2380"

13ETCD_ADVERTISE_CLIENT_URLS="https://192.168.18.148:2379"

14ETCD_INITIAL_CLUSTER="etcd01=https://192.168.18.128:2380,etcd02=https://192.168.18.148:2380,etcd03=https://192.168.18.145:2380"

15ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"

16ETCD_INITIAL_CLUSTER_STATE="new"

17

18[root@node1 ~]# systemctl start etcd

19[root@node1 ~]# systemctl status etcd

20● etcd.service - Etcd Server

21   Loaded: loaded (/usr/lib/systemd/system/etcd.service; disabled; vendor preset: disabled)

22   Active: active (running) since 三 2020-01-15 17:53:24 CST; 5s ago

23#状态为Active

24

25

Node2：7-5



1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
1`修改`

2[root@node2 ~]# systemctl stop firewalld.service

3[root@node2 ~]# setenforce 0

4[root@node2 ~]# vim /opt/etcd/cfg/etcd

5#[Member]

6ETCD_NAME="etcd03"

7ETCD_DATA_DIR="/var/lib/etcd/default.etcd"

8ETCD_LISTEN_PEER_URLS="https://192.168.18.145:2380"

9ETCD_LISTEN_CLIENT_URLS="https://192.168.18.145:2379"

10

11#[Clustering]

12ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.18.145:2380"

13ETCD_ADVERTISE_CLIENT_URLS="https://192.168.18.145:2379"

14ETCD_INITIAL_CLUSTER="etcd01=https://192.168.18.128:2380,etcd02=https://192.168.18.148:2380,etcd03=https://192.168.18.145:2380"

15ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"

16ETCD_INITIAL_CLUSTER_STATE="new"

17

18[root@node2 ~]# systemctl start etcd

19[root@node2 ~]# systemctl status etcd

20● etcd.service - Etcd Server

21   Loaded: loaded (/usr/lib/systemd/system/etcd.service; disabled; vendor preset: disabled)

22   Active: active (running) since 三 2020-01-15 17:55:24 CST; 5s ago

23 #状态为Active

24

25

群集状态验证：



1
2
3
4
5
6
7
8
9
10
1`回到7-3上输入以下命令：`

2[root@master k8s]# cd etcd-cert/

3[root@master etcd-cert]# /opt/etcd/bin/etcdctl --ca-file=ca.pem --cert-file=server.pem --key-file=server-key.pem --endpoints="https://192.168.18.128:2379,https://192.168.18.148:2379,https://192.168.18.145:2379" cluster-health

4member 9104d301e3b6da41 is healthy: got healthy result from https://192.168.18.148:2379

5member 92947d71c72a884e is healthy: got healthy result from https://192.168.18.145:2379

6member b2a6d67e1bc8054b is healthy: got healthy result from https://192.168.18.128:2379

7cluster is healthy

8`状态为healthy健康`

9

10

Related posts:

  1. Kubernetes之（二十）Helm程序包管理器
  2. Kubernetes入门
  3. Kubernetes之（四）kubeadm部署集群
  4. Nginx+keepalived 双机热备（主从模式）

给TA打赏
共{{data.count}}人
人已打赏
[db:标签]kube-proxyKuberneteslinuxrequest命令安装工具数据监控脚本
安全运维

WordPress网站专用docker容器环境带Waf

2020-7-18 20:04:44

安全运维

运维安全-Gitlab管理员权限安全思考

2021-9-19 9:16:14

猜你喜欢

Kubernetes入门

029
❯

解锁会员权限

开通会员

解锁海量优质VIP资源

立刻开通
个人中心
购物车
优惠劵
今日签到
有新私信 私信列表
搜索

  • 扫码打开当前页

返回顶部