首页 > 代码库 > Nginx+keepalived+Tomcat集群架构
Nginx+keepalived+Tomcat集群架构
keepalived实现nginx发生故障时，自动切换，实现nginx反向代理的高可用。
环境：
CentOS6.4 64bit；
Jdk6.tar.gz
Aapache-tomcat-6.0.41.tar.gz
nginx-1.4.2.tar.gz
主机规划：
keepalived-master:192.168.115.10
keepalived-slave: 192.168.115.20
tomcat1 : 192.168.115.30
tomcat2 : 192.168.115.40
JDK安装：
tar zxvf JDK6.tar.gz
vi /etc/profile.d/jdk.sh
export JAVA_HOME=/usr/local/jdk6
export CALASS_PATH=$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export JAVA_BIN=$JAVA_HOME/bin
export PATH=$PATH:$JAVA_HOME/bin
Tomcat安装：
tar zxvf Aapache-tomcat-6.0.41.tar.gz
mv ./tomcat /usr/local/tomcat1
cp -r /usr/local/tomcat1 /usr/local/tomcat2
修改tomcat2中server.xml定义的三个端口：
shutdown端口"8005
ajp端口： 8009
http端口： 8080
nginx的安装：
tar zxvf pcre-8.12.zip
./configure
make && make install
yum groupinstall "Development Tools"
yum install openssl openssl-devel
tar zxvf nginx-1.4.2.tar.gz
cd nginx-1.4.2
./configure --prefix=/usr --sbin-path=/usr/sbin/nginx --conf-path=/etc/nginx/nginx.conf --pid-path=/var/run/nginx.pid --lock-path=/var/lock/nginx.lock --user=nginx --group=nginx --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --with-http_ssl_module --with-http_stub_status_module --with-http_gzip_static_module --with-pcre
make
make install
#########Nginx启动脚本####
#!/bin/sh
#
# nginx - this script starts and stops the nginx daemon
#
# chkconfig: - 85 15
# description: Nginx is an HTTP(S) server, HTTP(S) reverse \
# proxy and IMAP/POP3 proxy server
# processname: nginx
# config: /etc/nginx/nginx.conf
# config: /etc/sysconfig/nginx
# pidfile: /var/run/nginx.pid
# Source function library.
. /etc/rc.d/init.d/functions
# Source networking configuration.
. /etc/sysconfig/network
# Check that networking is up.
[ "$NETWORKING" = "no" ] && exit 0
nginx="/usr/sbin/nginx"
prog=$(basename $nginx)
NGINX_CONF_FILE="/etc/nginx/nginx.conf"
[ -f /etc/sysconfig/nginx ] && . /etc/sysconfig/nginx
lockfile=/var/lock/subsys/nginx
make_dirs() {
# make required directories
user=`nginx -V 2>&1 | grep "configure arguments:" | sed ‘s/[^*]*--user=\([^ ]*\).*/\1/g‘ -`
options=`$nginx -V 2>&1 | grep ‘configure arguments:‘`
for opt in $options; do
if [ `echo $opt | grep ‘.*-temp-path‘` ]; then
value=http://www.mamicode.com/`echo $opt | cut -d"=" -f 2`
if [ ! -d "$value" ]; then
# echo "creating" $value
mkdir -p $value && chown -R $user $value
fi
fi
done
}
start() {
[ -x $nginx ] || exit 5
[ -f $NGINX_CONF_FILE ] || exit 6
make_dirs
echo -n $"Starting $prog: "
daemon $nginx -c $NGINX_CONF_FILE
retval=$?
echo
[ $retval -eq 0 ] && touch $lockfile
return $retval
}
stop() {
echo -n $"Stopping $prog: "
killproc $prog -QUIT
retval=$?
echo
[ $retval -eq 0 ] && rm -f $lockfile
return $retval
}
restart() {
configtest || return $?
stop
sleep 1
start
}
reload() {
configtest || return $?
echo -n $"Reloading $prog: "
killproc $nginx -HUP
RETVAL=$?
echo
}
force_reload() {
restart
}
configtest() {
$nginx -t -c $NGINX_CONF_FILE
}
rh_status() {
status $prog
}
rh_status_q() {
rh_status >/dev/null 2>&1
}
case "$1" in
start)
rh_status_q && exit 0
$1
;;
stop)
rh_status_q || exit 0
$1
;;
restart|configtest)
$1
;;
reload)
rh_status_q || exit 7
$1
;;
force-reload)
force_reload
;;
status)
rh_status
;;
condrestart|try-restart)
rh_status_q || exit 0
;;
*)
echo $"Usage: $0 {start|stop|status|restart|condrestart|try-restart|reload|force-reload|configtest}"
exit 2
esac
#########Nginx启动脚本####
安装keepalived
yum install popt-devel
tar zxvf keepalived-1.1.19.tar.gz
./configure --prefix=/usr/local/keepalived
make && make install
规划配置：
mkdir /etc/keepalived
cp /usr/local/keepalived/sbin/keepalived /usr/sbin/
cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d
cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/
修改配置文件：
! Configuration File for keepalived
global_defs {
notification_email {
1075841124@qq.com
}
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_instance VI_1 {
state MASTER 主为MASTER。备为BACKUP
interface eth0
virtual_router_id 51 主和备相同
priority 100 主高备低
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.115.222
192.168.115.111
192.168.115.99
}
}
观察:
MASTER:
/etc/init.d/keepalived start
ip addr show看到如下信息：
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:7e:20:10 brd ff:ff:ff:ff:ff:ff
inet 192.168.115.10/24 brd 192.168.115.255 scope global eth0
inet 192.168.115.222/32 scope global eth0
inet 192.168.115.111/32 scope global eth0
inet 192.168.115.99/32 scope global eth0
inet6 fe80::20c:29ff:fe7e:2010/64 scope link
valid_lft forever preferred_lft forever
BACKUP:
/etc/init.d/keepalived start
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:05:ff:b3 brd ff:ff:ff:ff:ff:ff
inet 192.168.115.20/24 brd 192.168.115.255 scope global eth0
inet6 fe80::20c:29ff:fe05:ffb3/64 scope link
valid_lft forever preferred_lft forever
停止master上的keepalived后在观察backup：
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:05:ff:b3 brd ff:ff:ff:ff:ff:ff
inet 192.168.115.20/24 brd 192.168.115.255 scope global eth0
inet 192.168.115.222/32 scope global eth0
inet 192.168.115.111/32 scope global eth0
inet 192.168.115.99/32 scope global eth0
inet6 fe80::20c:29ff:fe05:ffb3/64 scope link
valid_lft forever preferred_lft forever
配置nginx代理tomcat
http模块中添加：
upstream tomcat {
server 192.168.115.30:8080 weight=3;
server 192.168.115.30:8081 weight=3;
server 192.168.115.40:8080 weight=3;
server 192.168.115.40:8081 weight=3;
location / {
proxy_pass http://tomcat;
}
Nginx配置完毕。此时测试：
192.168.115.10：MASTER
/etc/init.d/keepalived start
server nginx start
192.168.115.20:SLAVE
/etc/init.d/keepalived start
server nginx start
后端tomcat全部打开。
注意此时，在keepalived上配置的虚拟IP是：
192.168.115.222
192.168.115.111
192.168.115.99
在浏览器中访问任何一个VIP，nginx都能代理到后方。并且能正常访问到TOMCAT主页面。
现在把MASTER上的nginx stop，观察
192.168.115.10 MASTER：
service nginx stop
ip addr show 观察：发现VIP消失
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:7e:20:10 brd ff:ff:ff:ff:ff:ff
inet 192.168.115.10/24 brd 192.168.115.255 scope global eth0
inet6 fe80::20c:29ff:fe7e:2010/64 scope link
valid_lft forever preferred_lft forever
于是观察BACKUP上，ip addr show 观察已经获取VIP了。
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:05:ff:b3 brd ff:ff:ff:ff:ff:ff
inet 192.168.115.20/24 brd 192.168.115.255 scope global eth0
inet 192.168.115.222/32 scope global eth0
inet 192.168.115.111/32 scope global eth0
inet 192.168.115.99/32 scope global eth0
inet6 fe80::20c:29ff:fe05:ffb3/64 scope link
valid_lft forever preferred_lft forever
再用浏览器访问VIP，看是否能问到tomcat页面。
依然能访问到，说明keepalived起到作用了。
这就是keepalived双机热备，故障转移的表现功能所在。在Nginx+Keepalived的架构上，nginx双机实现了反向代理的高可用。一台nginx挂掉后，依然不会影响访问。前端负载均衡已经解决了，但是后端的tomcat服务器挂掉后，该如何呢？keepalived是无法做到对后端服务器的健康检测的。这需要Nginx的功能。
本文出自 “常用文档” 博客，请务必保留此出处http://yujianglei.blog.51cto.com/7215578/1566730
Nginx+keepalived+Tomcat集群架构