我虚拟了三台centos7,要将one(192.168.56.31)、two(192.168.56.32)配置成nginx+keepalived双机热备(思路是当一台服务器的nginx挂掉时,可以自动用另一台的nginx,就是当主服务器nginx的端口不通时,自动关闭自己的keepalived服务,这样虚拟ip就会指到备用ip上),three(192.168.56.33)做tomcat服务器。实验环境下先把防火墙和selinux关闭:setenforce 0 && systemctl stop firewalld
yum -y install epel-release && yum -y install nginx
。然后再来安装一下keepalived:yum -y install keepalived
。安装完成后可以使用keepalived --help
来进行keepalived的命令帮助。两者的配置文件都在etc目录下。再将jdk和tomcat上传到three中并安装,我这设置了三个tomcat实例,端口分别为:8180、8280、8380。 然后先来配置nginx,配置文件在/etc/nginx/中: # For more information on configuration, see:# * Official English Documentation: http://nginx.org/en/docs/# * Official Russian Documentation: http://nginx.org/ru/docs/user nginx; #运行用户,避免权限问题,就用root吧worker_processes auto; #工作进程数,表示自动选择,cpu多的话,可以手动设置一般为cpu的倍数。error_log /var/log/nginx/error.log; #错误日志路径pid /run/nginx.pid; #PID文件路径# Load dynamic modules. See /usr/share/nginx/README.dynamic. include /usr/share/nginx/modules/*.conf; #包含这个文件的信息events { worker_connections 1024; #工作连接数,指一个进程可以产生多少个连接,用 ulimit -n 来查看,但是这是理论上,没必要设置这么大}http { log_format main '$remote_addr - $remote_user [$time_local] "$request" ' #定义访问日志的显示格式 '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"'; access_log /var/log/nginx/access.log main; #访问日志的路径 sendfile on; tcp_nopush on; tcp_nodelay on; keepalive_timeout 65; types_hash_max_size 2048; include /etc/nginx/mime.types; #将指定的配置文件引入到这个文件中来 default_type application/octet-stream; # Load modular configuration files from the /etc/nginx/conf.d directory. # See http://nginx.org/en/docs/ngx_core_module.html#include # for more information. include /etc/nginx/conf.d/*.conf; server { #服务器设置,可以设置多个服务器 listen 80 default_server; #设置监听的端口 listen [::]:80 default_server; server_name _; #服务器名称 root /usr/share/nginx/html; #服务器默认网站的目录 # Load configuration files for the default server block. include /etc/nginx/default.d/*.conf; location / { #设置请求转向 proxy_pass http://cs; proxy_set_header HOST $host; #不添加下面三行时,有次转发后点击连接前面域名会变为cs proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; } error_page 404 /404.html; location = /40x.html { } error_page 500 502 503 504 /50x.html; location = /50x.html { } }upstream cs { #这是一个反向连接池,指向了三个tomcat实例 server 192.168.56.33:8180; server 192.168.56.33:8280; server 192.168.56.33:8380;}# Settings for a TLS enabled server.## server {# listen 443 ssl http2 default_server;# listen [::]:443 ssl http2 default_server;# server_name _;# root /usr/share/nginx/html;## ssl_certificate "/etc/pki/nginx/server.crt";# ssl_certificate_key "/etc/pki/nginx/private/server.key";# ssl_session_cache shared:SSL:1m;# ssl_session_timeout 10m;# ssl_ciphers HIGH:!aNULL:!MD5;# ssl_prefer_server_ciphers on;## # Load configuration files for the default server block.# include /etc/nginx/default.d/*.conf;## location / {# }## error_page 404 /404.html;# location = /40x.html {# }## error_page 500 502 503 504 /50x.html;# location = /50x.html {# }# }}
配置完成后把,nginx和三个tomcat服务起来后,就能成功访问了,接下来我们再来配置一下keepalived,配置文件keeplived.conf在安装好后的/etc/keeplived中,下面是配置文件的解释:
! Configuration File for keepalivedglobal_defs { #全局定义块 notification_email { #指定keepalived在发生切换时需要发送email到的对象,一行一个。 acassen@firewall.loc failover@firewall.loc sysadmin@firewall.loc } notification_email_from Alexandre.Cassen@firewall.loc #发件人是谁 smtp_server 192.168.200.1 #指定smtp服务器地址 smtp_connect_timeout 30 #指定smtp连接超时时间 router_id LVS_DEVEL #Lvs负载均衡器标识(lvs_id)。在一个网络内,它应该是唯一的。 vrrp_skip_check_adv_addr vrrp_strict vrrp_garp_interval 0 vrrp_gna_interval 0}
vrrp_instance VI_1 { #虚拟vrrp设置模块 state MASTER #指定那个为master,那个为backup,如果设置了nopreempt这个值不起作用,主备靠priority决定。 interface eth0 #设置实例绑定的网卡 virtual_router_id 51 #虚拟路由的id priority 100 #优先级,高优先级竞选为master advert_int 1 #检查间隔,默认1秒 authentication { #认证设置 auth_type PASS #认证方式 auth_pass 1111 #认证密码 } virtual_ipaddress { #虚拟ip地址设置 192.168.200.16 192.168.200.17 192.168.200.18 }}
#虚拟服务器virtual_server定义块 ,该部分是用来管理LVS的,是实现keepalive和LVS相结合的模块。ipvsadm命令可以实现的管理在这里都可以通过参数配置实现。virtual_server 192.168.200.100 443 { #虚拟IP地址,要和vrrp_instance模块中的virtual_ipaddress地址一致 delay_loop 6 #健康检查时间间隔 lb_algo rr #lvs调度算法有rr|wrr|lc|wlc|lblc|sh|dh lb_kind NAT #负载均衡转发规则NAT|DR|RUN persistence_timeout 50 #会话保持时间 protocol TCP #使用的协议 real_server 192.168.201.100 443 { #真实ip地址 weight 1 #默认为1,0为失效 SSL_GET { url { path / digest ff20ad2481f97b1754ef3e12ecd3a9cc } url { path /mrtg/ digest 9b3a0c85a887a256d6939da88aabd8cd } connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } }}
virtual_server 10.10.10.2 1358 { delay_loop 6 lb_algo rr lb_kind NAT persistence_timeout 50 protocol TCP sorry_server 192.168.200.200 1358 real_server 192.168.200.2 1358 { weight 1 HTTP_GET { url { path /testurl/test.jsp digest 640205b7b0fc66c1ea91c463fac6334d } url { path /testurl2/test.jsp digest 640205b7b0fc66c1ea91c463fac6334d } url { path /testurl3/test.jsp digest 640205b7b0fc66c1ea91c463fac6334d } connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } } real_server 192.168.200.3 1358 { weight 1 HTTP_GET { url { path /testurl/test.jsp digest 640205b7b0fc66c1ea91c463fac6334c } url { path /testurl2/test.jsp digest 640205b7b0fc66c1ea91c463fac6334c } connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } }}virtual_server 10.10.10.3 1358 { delay_loop 3 lb_algo rr lb_kind NAT persistence_timeout 50 protocol TCP real_server 192.168.200.4 1358 { weight 1 HTTP_GET { url { path /testurl/test.jsp digest 640205b7b0fc66c1ea91c463fac6334d } url { path /testurl2/test.jsp digest 640205b7b0fc66c1ea91c463fac6334d } url { path /testurl3/test.jsp digest 640205b7b0fc66c1ea91c463fac6334d } connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } } real_server 192.168.200.5 1358 { weight 1 HTTP_GET { url { path /testurl/test.jsp digest 640205b7b0fc66c1ea91c463fac6334d } url { path /testurl2/test.jsp digest 640205b7b0fc66c1ea91c463fac6334d } url { path /testurl3/test.jsp digest 640205b7b0fc66c1ea91c463fac6334d } connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } }}
配置文件有点多,我们用不到的功能可以不往上配,这样能使性能更好!我的脚本如下:
global_defs { notification_email { 652179279@qq.com } notification_email_from smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id one}vrrp_script chk_nginx { #因为要检测nginx服务状态,所以创建一个检查脚本 script "/usr/local/check_ng.sh" interval 3}vrrp_instance VI_1 { state MASTER interface enp0s8 virtual_router_id 60 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.56.100 } track_script { chk_nginx }}
接下来就是去上面配置的路径创建脚本了vi /usr/local/check_ng.sh
:
#!/bin/bashd=`date --date today +%Y%m%d_%H:%M:%S`n=`ps -C nginx --no-heading|wc -l`if [ $n -eq "0" ]; then systemctl start nginx n2=`ps -C nginx --no-heading|wc -l` if [ $n2 -eq "0" ]; then echo "$d nginx down,keepalived will stop" >> /var/log/check_ng.log systemctl stop keepalived fifi
然后增加执行权限。再配置备用机:
global_defs { notification_email { 652179279@qq.com } notification_email_from smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id two}vrrp_script chk_nginx { script "/usr/local/sbin/check_ng.sh" interval 3}vrrp_instance VI_1 { state BACKUP interface enp0s8 virtual_router_id 60 priority 90 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.56.100 } track_script { chk_nginx }}
配置检测脚本和配置主机一样!生产环境下需要开放112端口,是keepalived的!