架构设计:负载均衡层设计方案(7)——LVS + Keepalived + Nginx安装及配置

释放双眼,带上耳机,听听看~!

1、概述

上篇文章《架构设计:负载均衡层设计方案(6)——Nginx + Keepalived构建高可用的负载层》(http://www.aiuxian.com/article/p-2765781.html) 我们讲解了Nginx的故障切换,并且承诺各位读者会尽快讲解 LVS + Keepalived + Nginx的安装和配置。在中间由于工作的原因,我又插写了三篇关于zookeeper的原理使用的文章。今天这边文章我们回归主题,为各位读者讲解LVS + Keepalived + Nginx的安装及配置。

链接地址2、安装计划和准备工作

下图,我们表示了本篇文章要搭建的整个集成架构的抽象结构:

我们采用两个LVS节点(141和142),但是一个时间工作的只有一个LVS节点,另一个始终处于热备standby状态,由keepalived监控这两个节点的工作状态并完成切换。

在LVS节点下,我们采用LVS-DR工作模式挂载了两个Nginx节点(131、132)。并最终将外网请求交由这两个节点进行处理。
注意:在实际工作中,Nginx下面一般就是访问静态资源、动态资源的配置了。

链接地址2-1、准备两个keepalived节点

首先我们在将要安装LVS的两个节点上,先安装keepalived,并保证这两个keepalived节点能够正常工作(监控批次的状态)。当然,您也可以先准备LVS,在准备keepalived。

我想准备keepalived节点,大家应该轻车熟路了吧,在http://www.aiuxian.com/article/p-2765781.htmlt5%20%E6%9E%B6%E6%9E%84%E8%AE%BE%E8%AE%A1%EF%BC%9A%E8%B4%9F%E8%BD%BD%E5%9D%87%E8%A1%A1%E5%B1%82%E8%AE%BE%E8%AE%A1%E6%96%B9%E6%A1%88%EF%BC%886%EF%BC%89%E2%80%94%E2%80%94Nginx%20+%20Keepalived%E6%9E%84%E5%BB%BA%E9%AB%98%E5%8F%AF%E7%94%A8%E7%9A%84%E8%B4%9F%E8%BD%BD%E5%B1%82这篇文章中详细介绍了keepalived的最简配置方式。为了大家阅读方便,我们在这里再进行依次简要说明。准备keepalived的整个过程包括:

  1. 安装必要的支撑组件,源码安装keepalived
  2. 将keepalived注册成节点的服务,以便保证keepalived在节点启动时就开始工作
  3. 更改keepalived的配置文件,让其可以正常工作
  4. 验证准备工作

=============安装keepalived


1
2
3
4
5
6
1<code class="hljs ruby has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background-position: initial initial; background-repeat: initial initial;">[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@lvs1</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># yum install -y zlib zlib-devel gcc gcc-c++ openssl openssl-devel openssh</span>
2[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@lvs1</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># tar -zxvf keepalived-1.2.17.tar.gz</span>
3[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@lvs1</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># cd keepalived-1.2.17</span>
4[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@lvs1</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># ./configure --perfix=/usr/keepalived-1.2.17</span>
5[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@lvs1</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># make & make install </span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li></ul>
6

=============将keepalived注册成服务(如果您使用的默认路径安装,就不需要cp命令了)


1
2
3
4
5
6
7
8
1<code class="hljs ruby has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background-position: initial initial; background-repeat: initial initial;">[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@lvs1</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># cp /usr/keepalived-1.2.17/etc/sysconfig/keepalived  /etc/sysconfig/keepalived </span>
2[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@lvs1</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># cp /usr/keepalived-1.2.17/sbin/keepalived /usr/sbin/keepalived</span>
3[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@lvs1</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># cp /usr/keepalived-1.2.17/etc/rc.d/init.d/keepalived  /etc/rc.d/init.d/keepalived</span>
4[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@lvs1</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># mkdir /etc/keepalived</span>
5[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@lvs1</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># cp /usr/keepalived-1.2.17/etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf</span>
6[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@lvs1</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># 可以做成服务了(不要拷贝,没用的)</span>
7[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@lvs1</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># chkconfig keepalived on</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li></ul>
8

做成服务后,先不要急着启动,因为配置文件还没有改好。 
========配置keepalived(配置文件在:/etc/keepalived/keepalived.conf)


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
1<code class="hljs ruby has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background-position: initial initial; background-repeat: initial initial;">! <span class="hljs-constant" style="box-sizing: border-box;">Configuration</span> <span class="hljs-constant" style="box-sizing: border-box;">File</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> keepalived
2
3global_defs {
4   <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#notification_email {</span>
5   <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#  acassen<span class="hljs-yardoctag" style="color: rgb(102, 0, 102); box-sizing: border-box; font-weight: bold;">@firewall</span>.loc</span>
6   <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#  failover<span class="hljs-yardoctag" style="color: rgb(102, 0, 102); box-sizing: border-box; font-weight: bold;">@firewall</span>.loc</span>
7   <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#  sysadmin<span class="hljs-yardoctag" style="color: rgb(102, 0, 102); box-sizing: border-box; font-weight: bold;">@firewall</span>.loc</span>
8   <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#}</span>
9   <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#notification_email_from Alexandre.Cassen<span class="hljs-yardoctag" style="color: rgb(102, 0, 102); box-sizing: border-box; font-weight: bold;">@firewall</span>.loc</span>
10   <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#smtp_server 192.168.200.1</span>
11   <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#smtp_connect_timeout 30</span>
12   router_id <span class="hljs-constant" style="box-sizing: border-box;">LVS_DEVEL</span>
13}
14
15vrrp_instance <span class="hljs-constant" style="box-sizing: border-box;">VI_1</span> {
16    <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#141节点设置为MASTER,142或者还有其他的节点设置为BACKUP</span>
17    <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#还记得我们前面文章讲到的无抢占设置吗?这里也可以用哦。</span>
18    state <span class="hljs-constant" style="box-sizing: border-box;">MASTER</span>
19    <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#网络适配器名称</span>
20    interface eth<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>
21    virtual_router_id <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">51</span>
22    <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#所有的SLAVE节点的优先级都要比这个设置值低</span>
23    priority <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">120</span>
24    advert_int <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>
25    <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#真实ip,142要改成相应的lvs节点真实ip</span>
26    mcast_src_ip=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">220.141</span>
27    authentication {
28        auth_type <span class="hljs-constant" style="box-sizing: border-box;">PASS</span>
29        auth_pass <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1111</span>
30    }
31    <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#虚拟/浮动IP</span>
32    virtual_ipaddress {
33        <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">220.140</span>
34    }
35}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li></ul>
36

以上配置还是最简单的keepalived配置,因为我们还没有加上配合LVS使用的虚拟ip监测设置和下层真实ip监测的设置。最简配置主要是为了保证keepalived节点是工作正常的。

将以上的配置分别对应到LVS的两个节点(注意要改动的地方哦)

==========进行keepalived工作状态的检查:


1
2
1<code class="hljs ruby has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background-position: initial initial; background-repeat: initial initial;">[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@lvs1</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># /etc/init.d/keepalived start</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>
2

现在设置为MASTER的keepalived节点(或者在非抢占模式下,优先级最高的那个节点),已经绑定了140这个虚拟ip了:


1
2
3
4
5
6
7
8
9
10
11
12
13
1<code class="hljs perl has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background-position: initial initial; background-repeat: initial initial;">[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@lvs2</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># ip addr</span>
2<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>: lo: <LOOPBACK,UP,LOWER_UP> mtu <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16436</span> qdisc noqueue <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">state</span> UNKNOWN
3    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">link</span>/loopback <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">00</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">00</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">00</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">00</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">00</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">00</span> brd <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">00</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">00</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">00</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">00</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">00</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">00</span>
4    inet <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">127.0</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8</span> scope host lo
5    inet6 ::<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">128</span> scope host
6       valid_lft forever preferred_lft forever
7<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>: eth<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1500</span> qdisc pfifo_fast <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">state</span> UP qlen <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1000</span>
8    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">link</span>/ether <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">00</span>:0c:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">29</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">39</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">75</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">9</span>f brd ff:ff:ff:ff:ff:ff
9    inet <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">220.141</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">24</span> brd <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">220.255</span> scope global eth<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>
10    inet <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">220.140</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">32</span> scope global eth<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>
11    inet6 fe8<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>::<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">20</span>c:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">29</span>ff:fe39:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">759</span>f/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">64</span> scope <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">link</span>
12       valid_lft forever preferred_lft forever</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li></ul>
13

当然您也可以通过 /var/log/message的日志进行keepalived是否正常工作的验证。

链接地址2-2、继续两个keepalived节点上准备LVS

准备lvs的工作就太简单了,因为centos6.4、6.5、6.6都已经集成了LVS的核心,我们只需要安装LVS的管理工具就行了:

两个LVS节点都执行:


1
2
1<code class="hljs cmake has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background-position: initial initial; background-repeat: initial initial;">yum -y <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">install</span> ipvsadm</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>
2

还记得lvs管理工具怎么使用吗?请参见我介绍LVS单节点安装的博文:《架构设计:负载均衡层设计方案(5)——LVS单节点安装》。这里为了方便阅读,给出主要参数的含义列表:


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
1<code class="hljs haml has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background-position: initial initial; background-repeat: initial initial;">-<span class="ruby" style="box-sizing: border-box;"><span class="hljs-constant" style="box-sizing: border-box;">A</span> –add-service 在内核的虚拟服务器表中添加一条新的虚拟服务器记录。也就是增加一台新的虚拟服务器。
2</span>-<span class="ruby" style="box-sizing: border-box;"><span class="hljs-constant" style="box-sizing: border-box;">E</span> –edit-service 编辑内核虚拟服务器表中的一条虚拟服务器记录。
3</span>-<span class="ruby" style="box-sizing: border-box;"><span class="hljs-constant" style="box-sizing: border-box;">D</span> –delete-service 删除内核虚拟服务器表中的一条虚拟服务器记录。
4</span>-<span class="ruby" style="box-sizing: border-box;"><span class="hljs-constant" style="box-sizing: border-box;">C</span> –clear 清除内核虚拟服务器表中的所有记录。
5</span>-<span class="ruby" style="box-sizing: border-box;"><span class="hljs-constant" style="box-sizing: border-box;">R</span> –restore 恢复虚拟服务器规则
6</span>-<span class="ruby" style="box-sizing: border-box;"><span class="hljs-constant" style="box-sizing: border-box;">S</span> –save 保存虚拟服务器规则,输出为-<span class="hljs-constant" style="box-sizing: border-box;">R</span> 选项可读的格式
7</span>-<span class="ruby" style="box-sizing: border-box;">a –add-server 在内核虚拟服务器表的一条记录里添加一条新的真实服务器记录。也就是在一个虚拟服务器中增加一台新的真实服务器
8</span>-<span class="ruby" style="box-sizing: border-box;">e –edit-server 编辑一条虚拟服务器记录中的某条真实服务器记录
9</span>-<span class="ruby" style="box-sizing: border-box;">d –delete-server 删除一条虚拟服务器记录中的某条真实服务器记录
10</span>-<span class="ruby" style="box-sizing: border-box;"><span class="hljs-constant" style="box-sizing: border-box;">L</span> –list 显示内核虚拟服务器表
11</span>-<span class="ruby" style="box-sizing: border-box;"><span class="hljs-constant" style="box-sizing: border-box;">Z</span> –zero 虚拟服务表计数器清零(清空当前的连接数量等)
12</span>–set tcp tcpfin udp 设置连接超时值
13–start-daemon 启动同步守护进程。他后面可以是master 或backup,用来说明LVS Router 是master 或是backup。在这个功能上也可以采用keepalived 的VRRP 功能。
14–stop-daemon 停止同步守护进程
15-<span class="ruby" style="box-sizing: border-box;">t –tcp-service service-address 说明虚拟服务器提供的是tcp 的服务[<span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">vip:</span>port] <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">or</span> [real-server-<span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">ip:</span>port]
16</span>-<span class="ruby" style="box-sizing: border-box;">u –udp-service service-address 说明虚拟服务器提供的是udp 的服务[<span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">vip:</span>port] <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">or</span> [real-server-<span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">ip:</span>port]
17</span>-<span class="ruby" style="box-sizing: border-box;">f –fwmark-service fwmark 说明是经过iptables 标记过的服务类型。
18</span>-<span class="ruby" style="box-sizing: border-box;">s –scheduler scheduler 使用的调度算法,选项:rr|wrr|lc|wlc|lblc|lblcr|dh|sh|sed|nq, 默认的调度算法是: wlc.
19</span>-<span class="ruby" style="box-sizing: border-box;">p –persistent [timeout] 持久稳固的服务。这个选项的意思是来自同一个客户的多次请求,将被同一台真实的服务器处理。timeout 的默认值为<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">300</span> 秒。
20</span>-<span class="ruby" style="box-sizing: border-box;"><span class="hljs-constant" style="box-sizing: border-box;">M</span> –netmask netmask persistent granularity mask
21</span>-<span class="ruby" style="box-sizing: border-box;">r –real-server server-address 真实的服务器[<span class="hljs-constant" style="box-sizing: border-box;">Real</span>-<span class="hljs-constant" style="box-sizing: border-box;">Server</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:port</span>]
22</span>-<span class="ruby" style="box-sizing: border-box;">g –gatewaying 指定<span class="hljs-constant" style="box-sizing: border-box;">LVS</span> 的工作模式为直接路由模式<span class="hljs-constant" style="box-sizing: border-box;">DR</span>模式(也是<span class="hljs-constant" style="box-sizing: border-box;">LVS</span>默认的模式)
23</span>-<span class="ruby" style="box-sizing: border-box;">i –ipip 指定<span class="hljs-constant" style="box-sizing: border-box;">LVS</span> 的工作模式为隧道模式
24</span>-<span class="ruby" style="box-sizing: border-box;">m –masquerading 指定<span class="hljs-constant" style="box-sizing: border-box;">LVS</span> 的工作模式为<span class="hljs-constant" style="box-sizing: border-box;">NAT</span> 模式
25</span>-<span class="ruby" style="box-sizing: border-box;">w –weight weight 真实服务器的权值
26</span>–mcast-interface interface 指定组播的同步接口
27–connection 显示LVS 目前的连接 如:ipvsadm -L -c
28–timeout 显示tcp tcpfin udp 的timeout 值 如:ipvsadm -L –timeout
29–daemon 显示同步守护进程状态
30–stats 显示统计信息
31–rate 显示速率信息
32–sort 对虚拟服务器和真实服务器排序输出
33–numeric -n 输出IP 地址和端口的数字形式</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li></ul>
34

到后面正式启动LVS的时候,就不要问我参数含义咯。^_^

链接地址2-3、准备两个Nginx节点并保证可用

在《架构设计:负载均衡层设计方案(5)——LVS单节点安装》(http://www.aiuxian.com/article/p-2765780.html) 这篇文章中,我们详细讲解了Nginx节点的准备工作,但是为了方便各位读者阅读,这里我们大致再讲一下。

Nginx节点的准备工作主要由以下步骤构成(这个不是本文的重点,点到即可):

  1. 安装Nginx(当然,正式系统中,还涉及到Nginx的参数调优,可以参见http://www.aiuxian.com/article/p-2765777.htmlt7%20%E6%9E%B6%E6%9E%84%E8%AE%BE%E8%AE%A1%EF%BC%9A%E8%B4%9F%E8%BD%BD%E5%9D%87%E8%A1%A1%E5%B1%82%E8%AE%BE%E8%AE%A1%E6%96%B9%E6%A1%88%EF%BC%882%EF%BC%89%E2%80%94%E2%80%94Nginx%E5%AE%89%E8%A3%85
  2. 打开Nginx所在服务器的“路由”功能、关闭“ARP查询”功能
  3. 将VIP:192.168.220.140 设置成Nginx所在节点的回环IP

=============安装Nginx


1
2
3
4
5
6
7
8
1<code class="hljs ruby has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background-position: initial initial; background-repeat: initial initial;">[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@vm1</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># yum -y install make zlib zlib-devel gcc gcc-c++ ssh libtool</span>
2[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@vm1</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># 下载nginx(别拷贝,不能执行的)</span>
3[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@vm1</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># 解压nginx(别拷贝,不能执行的)</span>
4[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@vm1</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># ./configure –prefix=/usr/nginx-1.8.0 </span>
5[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@vm1</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># make && make install </span>
6[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@vm1</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># 设置环境变量(别拷贝,不能执行的)</span>
7[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@vm1</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># 启动nginx</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li></ul>
8

=============打开Nginx所在服务器的“路由”功能、关闭“ARP查询”功能


1
2
3
4
5
1<code class="hljs mel has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background-position: initial initial; background-repeat: initial initial;">[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@vm1</span> ~]# echo <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"1"</span> >/<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">proc</span>/sys/net/ipv4/conf/lo/arp_ignore
2[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@vm1</span> ~]# echo <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"2"</span> >/<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">proc</span>/sys/net/ipv4/conf/lo/arp_announce
3[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@vm1</span> ~]# echo <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"1"</span> >/<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">proc</span>/sys/net/ipv4/conf/all/arp_ignore
4[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@vm1</span> ~]# echo <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"2"</span> >/<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">proc</span>/sys/net/ipv4/conf/all/arp_announce</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li></ul>
5

=============设置回环IP


1
2
3
1<code class="hljs ruby has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background-position: initial initial; background-repeat: initial initial;">[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@vm1</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># ifconfig lo:0 192.168.220.140 broadcast 192.168.220.140 netmask 255.255.255.255 up</span>
2[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@vm1</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># route add -host 192.168.220.140 dev lo:0</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li></ul>
3

两台Nginx的节点都按照这样的方法去设置。然后使用浏览器,看看这两个节点的Nginx是否工作正常:

=====

链接地址2-4、其他说明

keepalived和LVS是天生配合完美的一对,
LVS负责进行请求转发不负责任何节点的健康监测;keepalived负责监控整个环境中,包括虚拟ip,真实ip对应的下层节点的健康状态监测。

链接地址3、开始配置:LVS-DR工作模式

链接地址3-1、keepalived的更改——健康监测

首先我们要更改之前配置的“最简keepalived”配置,让keepalived能够对结构中所有节点和虚拟ip的健康状态进行监测。更改配置和含义如下:


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
1<code class="hljs ruby has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background-position: initial initial; background-repeat: initial initial;">! <span class="hljs-constant" style="box-sizing: border-box;">Configuration</span> <span class="hljs-constant" style="box-sizing: border-box;">File</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> keepalived
2
3global_defs {
4   <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#notification_email {</span>
5   <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#  acassen<span class="hljs-yardoctag" style="color: rgb(102, 0, 102); box-sizing: border-box; font-weight: bold;">@firewall</span>.loc</span>
6   <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#  failover<span class="hljs-yardoctag" style="color: rgb(102, 0, 102); box-sizing: border-box; font-weight: bold;">@firewall</span>.loc</span>
7   <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#  sysadmin<span class="hljs-yardoctag" style="color: rgb(102, 0, 102); box-sizing: border-box; font-weight: bold;">@firewall</span>.loc</span>
8   <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#}</span>
9   <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#notification_email_from Alexandre.Cassen<span class="hljs-yardoctag" style="color: rgb(102, 0, 102); box-sizing: border-box; font-weight: bold;">@firewall</span>.loc</span>
10   <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#smtp_server 192.168.200.1</span>
11   <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#smtp_connect_timeout 30</span>
12   router_id <span class="hljs-constant" style="box-sizing: border-box;">LVS_DEVEL</span>
13}
14
15vrrp_instance <span class="hljs-constant" style="box-sizing: border-box;">VI_1</span> {
16    <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#141节点设置为MASTER,142或者还有其他的节点设置为BACKUP</span>
17    <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#还记得我们前面文章讲到的无抢占设置吗?这里也可以用哦。</span>
18    state <span class="hljs-constant" style="box-sizing: border-box;">MASTER</span>
19    <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#网络适配器名称</span>
20    interface eth<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>
21    virtual_router_id <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">51</span>
22    <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#所有的SLAVE节点的优先级都要比这个设置值低</span>
23    priority <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">120</span>
24    advert_int <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>
25    <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#真实ip,142要改成相应的lvs节点真实ip</span>
26    mcast_src_ip=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">220.141</span>
27    authentication {
28        auth_type <span class="hljs-constant" style="box-sizing: border-box;">PASS</span>
29        auth_pass <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1111</span>
30    }
31    <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#虚拟/浮动IP</span>
32    virtual_ipaddress {
33        <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">220.140</span>
34    }
35}
36
37virtual_server <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">220.140</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">80</span> {
38    <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#健康时间检查,单位秒</span>
39    delay_loop <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6</span>
40    <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#负载均衡调度算法wlc|rr,和您将使用的LVS的调度算法保持原则一致</span>
41    lb_algo rr
42    <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#负载均衡转发规则 DR NAT TUN。和您将启动的LVS的工作模式设置一致</span>
43    lb_kind <span class="hljs-constant" style="box-sizing: border-box;">DR</span>
44    <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#虚拟地址的子网掩码</span>
45    nat_mask <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">255.255</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">255.0</span>
46    <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#会话保持时间,因为我们经常使用的是无状态的集群架构,所以这个设置可有可无</span>
47    <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#persistence_timeout 50</span>
48    <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#转发协议,当然是TCP</span>
49    protocol <span class="hljs-constant" style="box-sizing: border-box;">TCP</span>
50
51    <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#真实的下层Nginx节点的健康监测</span>
52    real_server <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">220.131</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">80</span> {
53        <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#节点权重,</span>
54        weight <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">10</span>
55        <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#设置检查方式,可以设置HTTP_GET | SSL_GET</span>
56        <span class="hljs-constant" style="box-sizing: border-box;">HTTP_GET</span> {
57            url {
58              path /
59              digest ff20ad2481f97b1754ef3e12ecd3a9cc
60            }
61            <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#超时时间,秒。如果在这个时间内没有返回,则说明一次监测失败</span>
62            connect_timeout <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span>
63            <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#设置多少次监测失败,就认为这个真实节点死掉了</span>
64            nb_get_retry <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span>
65            <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">#重试间隔</span>
66            delay_before_retry <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span>
67        }
68    }
69
70    real_server <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">192.168</span>.<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">220.132</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">80</span> {
71        weight <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">10</span>
72        <span class="hljs-constant" style="box-sizing: border-box;">HTTP_GET</span> {
73            url {
74              path /
75              digest <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">640205</span>b7b0fc66c1ea91c463fac6334d
76            }
77            connect_timeout <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span>
78            nb_get_retry <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span>
79            delay_before_retry <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span>
80        }
81    }
82}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li><li style="box-sizing: border-box; padding: 0px 5px;">38</li><li style="box-sizing: border-box; padding: 0px 5px;">39</li><li style="box-sizing: border-box; padding: 0px 5px;">40</li><li style="box-sizing: border-box; padding: 0px 5px;">41</li><li style="box-sizing: border-box; padding: 0px 5px;">42</li><li style="box-sizing: border-box; padding: 0px 5px;">43</li><li style="box-sizing: border-box; padding: 0px 5px;">44</li><li style="box-sizing: border-box; padding: 0px 5px;">45</li><li style="box-sizing: border-box; padding: 0px 5px;">46</li><li style="box-sizing: border-box; padding: 0px 5px;">47</li><li style="box-sizing: border-box; padding: 0px 5px;">48</li><li style="box-sizing: border-box; padding: 0px 5px;">49</li><li style="box-sizing: border-box; padding: 0px 5px;">50</li><li style="box-sizing: border-box; padding: 0px 5px;">51</li><li style="box-sizing: border-box; padding: 0px 5px;">52</li><li style="box-sizing: border-box; padding: 0px 5px;">53</li><li style="box-sizing: border-box; padding: 0px 5px;">54</li><li style="box-sizing: border-box; padding: 0px 5px;">55</li><li style="box-sizing: border-box; padding: 0px 5px;">56</li><li style="box-sizing: border-box; padding: 0px 5px;">57</li><li style="box-sizing: border-box; padding: 0px 5px;">58</li><li style="box-sizing: border-box; padding: 0px 5px;">59</li><li style="box-sizing: border-box; padding: 0px 5px;">60</li><li style="box-sizing: border-box; padding: 0px 5px;">61</li><li style="box-sizing: border-box; padding: 0px 5px;">62</li><li style="box-sizing: border-box; padding: 0px 5px;">63</li><li style="box-sizing: border-box; padding: 0px 5px;">64</li><li style="box-sizing: border-box; padding: 0px 5px;">65</li><li style="box-sizing: border-box; padding: 0px 5px;">66</li><li style="box-sizing: border-box; padding: 0px 5px;">67</li><li style="box-sizing: border-box; padding: 0px 5px;">68</li><li style="box-sizing: border-box; padding: 0px 5px;">69</li><li style="box-sizing: border-box; padding: 0px 5px;">70</li><li style="box-sizing: border-box; padding: 0px 5px;">71</li><li style="box-sizing: border-box; padding: 0px 5px;">72</li><li style="box-sizing: border-box; padding: 0px 5px;">73</li><li style="box-sizing: border-box; padding: 0px 5px;">74</li><li style="box-sizing: border-box; padding: 0px 5px;">75</li><li style="box-sizing: border-box; padding: 0px 5px;">76</li><li style="box-sizing: border-box; padding: 0px 5px;">77</li><li style="box-sizing: border-box; padding: 0px 5px;">78</li><li style="box-sizing: border-box; padding: 0px 5px;">79</li><li style="box-sizing: border-box; padding: 0px 5px;">80</li><li style="box-sizing: border-box; padding: 0px 5px;">81</li><li style="box-sizing: border-box; padding: 0px 5px;">82</li></ul>
83

这样一来,keepalived就可以检查整个架构中的所有节点状态了。
另外要说明的是,这个检查过程并不是必须的,您使用keepalived的最简配置也是可以的,不过您就需要自己写监测脚本了(道理是一样的)。

链接地址3-2、启动两个LVS节点

启动LVS的过程就太简单了(两个节点都是一样的启动方式):


1
2
3
4
5
6
7
1<code class="hljs ruby has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background-position: initial initial; background-repeat: initial initial;">[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@lvs2</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># ipvsadm -C</span>
2[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@lvs2</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># ipvsadm -At 192.168.220.140:80 -s rr </span>
3[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@lvs2</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># ipvsadm -at 192.168.220.140:80 -r 192.168.220.131 -g</span>
4[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@lvs2</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># ipvsadm -at 192.168.220.140:80 -r 192.168.220.132 -g</span>
5[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@lvs2</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># 然后我们可以使用ipvsadm 监控目前LVS的状态</span>
6[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@lvs2</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># ipvsadm</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li></ul>
7

注意:处于standby的lvs1节点也要这样进行设置。 

还有,以上的LVS的设置,和real server上的设置,在重启后都会消失,所以一定要做成脚本哦。

链接地址4、验证工作

这样LVS + Keepalived + Nginx方式的配置就做完了。现在我们进行搭建效果的监测:

链接地址4-1、验证Master-LVS节点的工作

我们使用两个不同的浏览器,验证Master-LVS节点的工作:

=========浏览器1: 

=========浏览器2: 

看来140这个VIP下的LVS工作是正常的。

链接地址4-2、验证Master-LVS节点停止后的效果

下面我们停止Master-LVS1:


1
2
1<code class="hljs ruby has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background-position: initial initial; background-repeat: initial initial;">[root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@lvs2</span> ~]<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># service keepalived stop</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>
2

在经历了一些访问停顿后,浏览器1显示的效果如下(这就是为什么keepalived最好设置为非抢占模式):

链接地址5、后文介绍

好了,到这里负载均衡层所使用的几个标准工具就介绍完了。下一篇文章我们将进行总结,然后进入架构设计:业务层设计方案 的系列文章。在下一个系列文章中,我们将介绍至少两套SOA的实现、至少两套系统间通信使用的消息队列。哦,应我朋友的要求,我会专门写几篇文章,介绍java中线程的基础知识和进阶知识。

给TA打赏
共{{data.count}}人
人已打赏
安全经验

独立博客怎样申请谷歌Adsense

2021-10-11 16:36:11

安全经验

安全咨询服务

2022-1-12 14:11:49

个人中心
购物车
优惠劵
今日签到
有新私信 私信列表
搜索