首页 > 代码库 > 利用lvs+keepalived实现高可用负载均衡环境的部署
利用lvs+keepalived实现高可用负载均衡环境的部署
此方案用于lvs+keepalived部署 ,本系统主要使用的技术有:ipvsadm、keepalived
一、服务器环境
ip地址分配如下:
lvs1(master) ip:192.168.30.209
lvs2(backup) ip:192.168.30.210
vip1:192.168.30.88
realserver1 ip:192.168.30.201:8087
192.168.30.201:8088
realserver2 ip:192.168.30.202:8087
192.168.30.202:8088
vip2:192.168.30.89
realserver3 ip:192.168.30.82:8080
realserver4 ip:192.168.30.83:8080
注:realserver端口选取需要选择正在使用的端口,不能随便选择。
版本信息及软件路径:
系统:CentOS6.6
ipvsadm:ipvsadm-2.16
keepalived:keepalived-1.2.20
ipvsadm包下载目录:/usr/local/src/
keepalived 配置文件:/etc/keepalived/keepalived.conf
realserver 启动脚本路径:/usr/local/bin/lvs_real.sh
lvs+keepalived网络拓扑图
二、应用安装
ipvsadm安装(在两台lvs均衡器上分别安装)
首先查看系统内核版本: uname –a 根据内核版本下载对应的安装包。
uname -a Linux wangwq01 2.6.32-504.el6.x86_64 #1 SMP Wed Oct 15 04:27:16 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
可知系统内核版本是2.6.32版本,故下载相对应的版本软件。
1.下载ipvsadm
cd /usr/local/src wget http://www.linuxvirtualserver.org/software/kernel-2.6/ipvsadm-1.26.tar.gz
2.做软链接,因为编译时会用到/usr/srx/linux,不做软链接会出现错误,kernel目录下是系统内核版本号。
ln -s /usr/src/kernels/2.6.32-431-el6.x86_64 /usr/src/linux
3.解压包
tar zxf ipvsadm-1.26.tar.gz
4.编译安装
cd ipvsadm-1.26 make && make install
make步骤执行完最好检查是否成功:echo $? 返回0即为成功。
如果有错,很可能是缺少依赖的包,安装即可,常见的包有:
yum install –y gcc yum install –y popt-devel yum install –y popt-static yum install –y libnl-devel
5.检查是否安装正确
ipvsadm
如果出现类似下面的内容,即安装正确。
IP VirtualServer version 1.2.1 (size=4096)
ProtLocalAddress:Port Scheduler Flags
->RemoteAddress:Port Forward Weight ActiveConn InActConn
6.检查当前加载的内核模块,看是否存在ip_vs模块
lsmod |grep ip_vs
正常情况可看到ip_vs模块。
Keepalived安装
A.lvs均衡器上安装与配置
1.下载keepalived
cd /usr/local/src wget http://www.keepalived.org/software/keepalived-1.2.20.tar.gz
2.解压包
tar -xzf keepalived-1.2.20.tar.gz
3.编译安装
cd keepalived-1.2.20 ./configure --prefix=/usr/local/keepalived
这个步骤可能会出错,原因是缺少相关的包,如:
!!! OpenSSL is not properly installed on your system. !!!
!!! Can not include OpenSSLheaders files.
这是因为缺少openssl的包:
#yum install -y openssl openssl-devel
还可能出现Please install libnfnetlinkheaders.这样的错,安装
#yum install –y libnfnetlink-devel
关于devel包和没有devel包的区别,看这个链接:
http://blog.csdn.net/crazyhacking/article/details/16808199
安装后重新运行,再看有没有别的错误,可根据最后的信息对应安装,直到运行成功。
make make install
注意这三个步骤执行完最好都用echo $?检查是否执行成功。
4.设置
ln -s /usr/local/keepalived/sbin/keepalived /usr/sbin/keepalived cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/ cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/ chmod 755 /etc/init.d/keepalived mkdir /etc/keepalived cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf
配置文件也可自己写,复制过去的只是模板,里面很多内容都是没用的。
5.修改配置文件
在lvs1(master)上配置:
vim /etc/keepalived/keepalived.conf #global define global_defs { router_id LVS_T1 #设置lvs的id,在一个网络内应该是唯一的 } vrrp_sync_group rx_group { #设置vrrp组,如果只有一台调度器,可不必设置 group { rx_one } } vrrp_instance rx_one { state MASTER #设置lvs的状态,报错MASTER和BACKUP两种,必须大写 interface eth0 #设置对外服务的接口 lvs_sync_daemon_interface eth0 #设置lvs监听的接口 virtual_router_id 30 #设置虚拟路由表示,并且同一个vrrp使用唯一的标识,MASTER和BACKUP的virtual_router_id是一致的 priority 150 #设置优先级,数值越大,优先级越高 advert_int 3 #设置同步时间间隔 authentication { #设置验证类型和密码 auth_type PASS auth_pass 1111 } virtual_ipaddress { #设置lvs vip 192.168.30.88 192.168.30.89 } } virtual_server 192.168.30.88 8087 { delay_loop 6 #健康检查时间间隔 lb_algo wrr #负载均衡调度算法,wrr为加权轮询 lb_kind DR #负载均衡转发规则 #persistence_timeout 0 #设置会话保持时间,对bbs等很有用 protocol TCP #协议 real_server 192.168.30.201 8087 { weight 3 #设置权重 TCP_CHECK { #tcp检查 connect_timeout 10 nb_get_retry 3 delay_before_retry 3 connect_port 8087 } } real_server 192.168.30.202 8087 { weight 3 #设置权重 TCP_CHECK { #tcp检查 connect_timeout 10 nb_get_retry 3 delay_before_retry 3 connect_port 8087 } } } virtual_server 192.168.30.888088 { delay_loop 6 lb_algo wrr lb_kind DR persistence_timeout 0 protocol TCP real_server 192.168.30.201 8088 { weight 2 TCP_CHECK { connect_timeout 10 nb_get_retry 3 delay_before_retry 3 connect_port 8088 } } real_server 192.168.30.202 8088 { weight 2 TCP_CHECK { connect_timeout 10 nb_get_retry 3 delay_before_retry 3 connect_port 8088 } } } virtual_server192.168.30.89 8080 { delay_loop 6 lb_algo wrr lb_kind DR persistence_timeout 0 protocol TCP real_server 192.168.30.82 8080 { weight 2 TCP_CHECK { connect_timeout 10 nb_get_retry 3 delay_before_retry 3 connect_port 8080 } } real_server 192.168.30.83 8080 { weight 2 TCP_CHECK { connect_timeout 10 nb_get_retry 3 delay_before_retry 3 connect_port 8080 } } }
在lvs2(backup)上:
vim /etc/keepalived/keepalived.conf #global define global_defs { router_id LVS_T2 } vrrp_sync_group rx_group { group { rx_one } } vrrp_instance rx_one { state BACKUP interface eth0 lvs_sync_daemon_interface eth0 virtual_router_id 30 priority 100 advert_int 3 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.30.88 192.168.30.89 } } virtual_server 192.168.30.88 8087 { delay_loop 6 lb_algo wrr lb_kind DR persistence_timeout 0 protocol TCP real_server 192.168.30.201 8087 { weight 2 TCP_CHECK { connect_timeout 10 nb_get_retry 3 delay_before_retry 3 connect_port 8087 } } real_server 192.168.30.202 8087 { weight 2 TCP_CHECK { connect_timeout 10 nb_get_retry 3 delay_before_retry 3 connect_port 8087 } } } virtual_server 192.168.30.88 8088 { delay_loop 6 lb_algo wrr lb_kind DR persistence_timeout 0 protocol TCP real_server 192.168.30.201 8088 { weight 2 TCP_CHECK { connect_timeout 10 nb_get_retry 3 delay_before_retry 3 connect_port 8088 } } real_server 192.168.30.202 8088 { weight 2 TCP_CHECK { connect_timeout 10 nb_get_retry 3 delay_before_retry 3 connect_port 8088 } } } virtual_server 192.168.30.89 8080 { delay_loop 6 lb_algo wrr lb_kind DR persistence_timeout 0 protocol TCP real_server 192.168.30.82 8080 { weight 2 TCP_CHECK { connect_timeout 10 nb_get_retry 3 delay_before_retry 3 connect_port 8080 } } real_server 192.168.30.83 8080 { weight 2 TCP_CHECK { connect_timeout 10 nb_get_retry 3 delay_before_retry 3 connect_port 8080 } } }
6.启动keepalived
分别在Master和Backup机上启动keepalived
/etc/init.d/keepalived start
可在日志里看到配置的信息:
tail -f /var/log/messages
B.Realserver上的配置
在realserver上写一个脚本,运行即可(注意脚本的VIP,不同的realserver对应不同的VIP)
vim /usr/local/bin/lvs_real.sh #! /bin/sh # descript :start RealServer. VIP=192.168.30.88 ./etc/rc.d/init.d/functions case"$1" in start) echo "start LVS of RealServer" /sbin/ifconfig lo:0 $VIP broadcast $VIP netmask 255.255.255.255 up echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce ;; stop) /sbin/ifconfig lo:0 down echo "close LVS of RealServer" echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce ;; *) echo "Usage : $0 {start|stop}" exit 1 esac
保存退出后,运行此脚本:
/usr/local/bin/lvs_real.sh start
然后查看网卡信息
ip addr
查看lo网卡是否有vip,如果有,则配置启动成功。
三、iptables设置
在两台lvs调度器和realserver上都开启相应的端口,本次使用的端口是8080、8087和8088,在realserver上只做对应端口的iptables即可。
在两台lvs均衡器上增加如下几条:
vim /etc/sysconfig/iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport 8080 -j ACCEPT -A INPUT -m state --state NEW -m tcp -p tcp --dport 8087 -j ACCEPT -A INPUT -m state --state NEW -m tcp -p tcp --dport 8088 -j ACCEPT -A INPUT -pvrrp -j ACCEPT :wq 保存退出 service iptables restart
Realserver上,查看有没有如下的规则,一般都是有的:
realserver1:
vim /etc/sysconfig/iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport 8087 -j ACCEPT -A INPUT -m state --state NEW -m tcp -p tcp --dport 8088 -j ACCEPT
realserver2:
vim /etc/sysconfig/iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport 8087 -j ACCEPT -A INPUT -m state --state NEW -m tcp -p tcp --dport 8088 -j ACCEPT
realserver3:
vim /etc/sysconfig/iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport 8080 -j ACCEPT
realserver4:
vim /etc/sysconfig/iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport 8080 -j ACCEPT
各自重启iptables:
service iptables restart
重启keepalived
/etc/init.d/keepalived restart
查看日志
tail -f /var/log/messages
看各自状态是否正常,再看ip信息
ip addr
master的网卡上会显示vip,backup不显示
测试:
使用两个终端,先动态查看backup的日志
tail -f /var/log/messages
再停掉master,看backup的日志变化,是否会自动切换成master,如果能,则正常;
再开启master,看backup的日志变化,是否会再次切换到backup,如果能,则正常。
在别的机器上使用telnet连接vip,在主调度器上用ipvsadm -ln命令查看请求是否正常轮询。
至此,lvs+keepalived的搭建配置基本完成。
本文出自 “我不是我” 博客,请务必保留此出处http://wangwq.blog.51cto.com/8711737/1852212
利用lvs+keepalived实现高可用负载均衡环境的部署