首页 > 代码库 > jumpserver篇--安装(高可用性 mariadb+haproxy)

jumpserver篇--安装(高可用性 mariadb+haproxy)

1. 需求

为了解决目前登陆方式多种多样,防火墙配置复杂,历史操作无记录,用户权限混乱等等

2. Jumpserver测试环境搭建

2.1. 环境

os:CentOS release 6.8 mini版本
JMS_Master      192.168.1.75    Centos-6.8-x86_64   mini
JMS_Slave        192.168.1.76    Centos-6.8-x86_64   mini
MySQL_Master    192.168.1.72   Centos-6.8-x86_64   mini
MySQL_Master    192.168.1.73    Centos-6.8-x86_64   mini
keepalive_VIP      192.168.1.74 
haproxy vip: 192.168.1。74 

2.2. 架构图

 技术分享

 

3. 相关软件安装

3.1. JDK 1.7安装

yum -y install java-1.7.0-openjdk  java-1.7.0-openjdk-devel

vim /etc/profile

最后增加两行

#JAVA_HOME根据不同的机器选择不同的路径即可

export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk.x86_64

export PATH=$PATH:$JAVA_HOME/bin

立即生效:source /etc/profile

javac命令确保可用

JAVA_HOME一定要配置好!!!用 echo $JAVA_HOME 命令验证!

 

 

3.2. mariadb Galera Cluster 安装

新增虚拟机具体配置配置MariaDByum

vim /etc/yum.repos.d/mariadb.repo

[mariadb]

name = MariaDB

baseurl = http://yum.mariadb.org/10.0/centos6-amd64

gpgkey=https://yum.mariadb.org/RPM-GPG-KEY-MariaDB

gpgcheck=1

yum install -y epel-release

 

yum update -y

yum -y install http://dl.fedoraproject.org/pub/epel/6/x86_64/socat-1.7.2.3-1.el6.x86_64.rpm

yum -y install MariaDB-Galera-server MariaDB-client rsync galerasystemctl enable  mariadb

 

---修改mariadb存储目录(不修改可忽略)

http://lddyw.blog.51cto.com/4151746/1684364

mkdir /home/data

cp -r /var/lib/mysql/ /home/data/

chown -R mysql:mysql /home/data/

vim /etc/my.cnf.d/server.cnf

[mariadb-10.0]

binlog_format=ROW

default-storage-engine=innodb

innodb_autoinc_lock_mode=2

innodb_locks_unsafe_for_binlog=1

query_cache_size=0

query_cache_type=0

bind-address=0.0.0.0

max_allowed_packet = 16M

datadir=/home/data/mysql

 

service mysql start

/usr/bin/mysql_secure_installation  --root 允许远程登陆 密码:jumpserverDb

[root@maria-server1 mysql]# mysql -u root -pjumpserverDb          

Welcome to the MariaDB monitor.  Commands end with ; or \g.

Your MariaDB connection id is 5

Server version: 5.5.40-MariaDB-wsrep MariaDB Server, wsrep_25.11.r4026

 

Copyright (c) 2000, 2015, Oracle, MariaDB Corporation Ab and others.

 

Type ‘help;‘ or ‘\h‘ for help. Type ‘\c‘ to clear the current input statement.

 

MariaDB [(none)]> grant all privileges on *.* to ‘root‘@‘%‘ identified by ‘jumpserverDb‘ with grant option;

flush privileges;

MariaDB [(none)]> select host, user from mysql.user;                                                  

+---------------+------+

| host          | user |

+---------------+------+

| %             | root |

| 127.0.0.1     | root |

| ::1           | root |

| localhost     |      |

| localhost     | root |

| maria-server1 |      |

| maria-server1 | root |

+---------------+------+

7 rows in set (0.00 sec)

3.3. 配置 MariaDB Galera Cluster

http://blog.sina.com.cn/s/blog_6de3aa8a0102w00d.html

---192.168.1.72

 

vim /etc/my.cnf.d/server.cnf

 

#

# * Galera-related settings

#

[galera]

# Mandatory settings

#wsrep_provider=

#wsrep_cluster_address=

#

# These groups are read by MariaDB server.

# Use it for options that only the server (but not clients) should see

#

# See the examples of server my.cnf files in /usr/share/mysql/

#

 

# this is read by the standalone daemon and embedded servers

[server]

 

# this is only for the mysqld standalone daemon

[mysqld]

 

#

# * Galera-related settings

#

[galera]

# Mandatory settings

#wsrep_provider=

#wsrep_cluster_address=

#binlog_format=row

#default_storage_engine=InnoDB

#innodb_autoinc_lock_mode=2

#bind-address=0.0.0.0

#

# Optional setting

#wsrep_slave_threads=1

#innodb_flush_log_at_trx_commit=0

 

# this is only for embedded server

[embedded]

 

# This group is only read by MariaDB servers, not by MySQL.

# If you use the same .cnf file for MySQL and MariaDB,

# you can put MariaDB-only options here

[mariadb]

 

# This group is only read by MariaDB-10.0 servers.

# If you use the same .cnf file for MariaDB of different versions,

# use this group for options that older servers don‘t understand

[mariadb-10.0]

 

binlog_format=ROW

default-storage-engine=innodb

innodb_autoinc_lock_mode=2

innodb_locks_unsafe_for_binlog=1

query_cache_size=0

query_cache_type=0

bind-address=0.0.0.0

datadir=/var/lib/mysql

innodb_log_file_size=100M

innodb_file_per_table

innodb_flush_log_at_trx_commit=2

wsrep_provider=/usr/lib64/galera/libgalera_smm.so

wsrep_cluster_address="gcomm:// 192.168.1.72, 192.168.1.73"

#wsrep_cluster_address="gcomm://"

wsrep_cluster_name=‘dbcluster‘

wsrep_node_address=‘192.168.1.72‘

wsrep_node_name=‘dbserver1‘

wsrep_sst_method=rsync

#wsrep_sst_auth=sst_user:dbpass

#

[mysqld_safe]

log-error=/var/log/mysqld.log

 

---192.168.1.73

 

vim /etc/my.cnf.d/server.cnf

[server]

 

# this is only for the mysqld standalone daemon

[mysqld]

 

#

# * Galera-related settings

#

[galera]

# Mandatory settings

#wsrep_provider=

#wsrep_cluster_address=

#binlog_format=row

#default_storage_engine=InnoDB

#innodb_autoinc_lock_mode=2

#bind-address=0.0.0.0

#

# Optional setting

#wsrep_slave_threads=1

#innodb_flush_log_at_trx_commit=0

 

# this is only for embedded server

[embedded]

 

# This group is only read by MariaDB servers, not by MySQL.

# If you use the same .cnf file for MySQL and MariaDB,

# you can put MariaDB-only options here

[mariadb]

 

# This group is only read by MariaDB-10.0 servers.

# If you use the same .cnf file for MariaDB of different versions,

# use this group for options that older servers don‘t understand

[mariadb-10.0]

 

binlog_format=ROW

default-storage-engine=innodb

innodb_autoinc_lock_mode=2

innodb_locks_unsafe_for_binlog=1

query_cache_size=0

query_cache_type=0

bind-address=0.0.0.0

datadir=/var/lib/mysql

innodb_log_file_size=100M

innodb_file_per_table

innodb_flush_log_at_trx_commit=2

wsrep_provider=/usr/lib64/galera/libgalera_smm.so

wsrep_cluster_address="gcomm:// 192.168.1.72, 192.168.1.73"

wsrep_cluster_name=‘dbcluster‘

wsrep_node_address=‘192.168.1.73‘

wsrep_node_name=‘dbserver2‘

wsrep_sst_method=rsync

[mysqld_safe]

log-error=/var/log/mysqld.log

 

wsrep_cluster_address ==注意一定要保证有节点存在

第一次初始化为wsrep_cluster_address= gcomm://

3.4. 查看集群状态

MariaDB [(none)]> show status like ‘%wsrep_%‘;

+------------------------------+-----------------------------------------+

| Variable_name                | Value                                   |

+------------------------------+-----------------------------------------+

| wsrep_local_state_uuid       | 2489b818-219b-11e6-9021-b61cb5e054fb    |

| wsrep_protocol_version       | 5                                       |

| wsrep_last_committed         | 18                                      |

| wsrep_replicated             | 0                                       |

| wsrep_replicated_bytes       | 0                                       |

| wsrep_repl_keys              | 0                                       |

| wsrep_repl_keys_bytes        | 0                                       |

| wsrep_repl_data_bytes        | 0                                       |

| wsrep_repl_other_bytes       | 0                                       |

| wsrep_received               | 3                                       |

| wsrep_received_bytes         | 238                                     |

| wsrep_local_commits          | 0                                       |

| wsrep_local_cert_failures    | 0                                       |

| wsrep_local_replays          | 0                                       |

| wsrep_local_send_queue       | 0                                       |

| wsrep_local_send_queue_avg   | 0.000000                                |

| wsrep_local_recv_queue       | 0                                       |

| wsrep_local_recv_queue_avg   | 0.000000                                |

| wsrep_local_cached_downto    | 18446744073709551615                    |

| wsrep_flow_control_paused_ns | 0                                       |

| wsrep_flow_control_paused    | 0.000000                                |

| wsrep_flow_control_sent      | 0                                       |

| wsrep_flow_control_recv      | 0                                       |

| wsrep_cert_deps_distance     | 0.000000                                |

| wsrep_apply_oooe             | 0.000000                                |

| wsrep_apply_oool             | 0.000000                                |

| wsrep_apply_window           | 1.000000                                |

| wsrep_commit_oooe            | 0.000000                                |

| wsrep_commit_oool            | 0.000000                                |

| wsrep_commit_window          | 1.000000                                |

| wsrep_local_state            | 4                                       |

| wsrep_local_state_comment    | Synced                                  |

| wsrep_cert_index_size        | 0                                       |

| wsrep_causal_reads           | 0                                       |

| wsrep_cert_interval          | 0.000000                                |

| wsrep_incoming_addresses     | 192.168.32.154:3306,192.168.32.153:3306 |

| wsrep_cluster_conf_id        | 24                                      |

| wsrep_cluster_size           | 2                                       |

| wsrep_cluster_state_uuid     | 2489b818-219b-11e6-9021-b61cb5e054fb    |

| wsrep_cluster_status         | Primary                                 |

| wsrep_connected              | ON                                      |

| wsrep_local_bf_aborts        | 0                                       |

| wsrep_local_index            | 0                                       |

| wsrep_provider_name          | Galera                                  |

| wsrep_provider_vendor        | Codership Oy <info@codership.com>       |

| wsrep_provider_version       | 3.5(rXXXX)                              |

| wsrep_ready                  | ON                                      |

| wsrep_thread_count           | 2                                       |

+------------------------------+-----------------------------------------+

48 rows in set (0.00 sec)

我们可以关注几个关键的参数:

 

wsrep_connected = on 链接已开启

 

wsrep_local_index = 1 在集群中的索引值

 

wsrep_cluster_size =3 集群中节点的数量

 

wsrep_incoming_addresses =192.168.32.154:3306,192.168.32.153:3306集群中节点的访问地址

3.5. 验证数据同步

maria-server1

MariaDB [(none)]> create database galera_test7;

MariaDB [(none)]> create database galera_test8;

MariaDB [(none)]> show databases;             

+--------------------+

| Database           |

+--------------------+

| information_schema |

| galera_test7       |

| galera_test8       |

| mysql              |

| performance_schema |

| test               |

+--------------------+

6 rows in set (0.00 sec)

 

maria-server2上查看

MariaDB [(none)]> show databases;            

+--------------------+

| Database           |

+--------------------+

| information_schema |

| galera_test7       |

| galera_test8       |

| mysql              |

| performance_schema |

| test               |

+--------------------+

6 rows in set (0.00 sec)

3.6. keepalived实现高可用

安装keepalived  --2个节点

yum -y install keepalived

---192.168.1.75

[root@maria-server1 keepalived]# vim /etc/keepalived/keepalived.conf

 

! Configuration File for keepalived

 

global_defs {

   notification_email {

     acassen@firewall.loc

     failover@firewall.loc

     sysadmin@firewall.loc

   }

   notification_email_from Alexandre.Cassen@firewall.loc

   smtp_server 127.0.0.1

   smtp_connect_timeout 30

   router_id LVS_DEVEL

}

 

vrrp_script check_haproxy {

    script "/etc/keepalived/check_haproxy.sh"

    interval 1

    weight 2

}

 

vrrp_instance VI_1 {

    state MASTER

    interface eth1

    virtual_router_id 88[i1] 

    priority 100[i2] 

    advert_int 1

    authentication {

        auth_type PASS

        auth_pass jumpserverPass

    }

    virtual_ipaddress {

        192.168.1.74

    }

    track_script {

       check_haproxy

     }

}

--集成haproxy  注意脚本权限

chmod +x /etc/keepalived/check_haproxy.sh

vim /etc/keepalived/check_haproxy.sh

A=`ps -C haproxy --no-header |wc -l`

if [ $A -eq 0 ]; then

    service haproxy  start

    sleep 2

    if [ `ps -C haproxy --no-header |wc -l` -eq 0 ]; then

             /etc/init.d/keepalived stop

    fi

fi

 

---192.168.1.76

[root@maria-server2 mysql]# vim /etc/keepalived/keepalived.conf

 

! Configuration File for keepalived

 

global_defs {

   notification_email {

     acassen@firewall.loc

     failover@firewall.loc

     sysadmin@firewall.loc

   }

   notification_email_from Alexandre.Cassen@firewall.loc

   smtp_server 127.0.0.1

   smtp_connect_timeout 30

   router_id LVS_DEVEL

}

 

vrrp_script check_haproxy {

    script "killall -0 haproxy"

    interval 1

    weight 2

}

 

vrrp_instance VI_1 {

    state BACKUP

    interface eth1

    virtual_router_id 88

    priority 99

    advert_int 1

    authentication {

        auth_type PASS

        auth_pass jumpserverPass

    }

    virtual_ipaddress {

        192.168.1.74

    }

    track_script {

       check_haproxy

     }

}

--集成haproxy  注意脚本权限

chmod +x /etc/keepalived/check_haproxy.sh

vim /etc/keepalived/check_haproxy.sh

A=`ps -C haproxy --no-header |wc -l`

if [ $A -eq 0 ]; then

    service haproxy  start

    sleep 2

    if [ `ps -C haproxy --no-header |wc -l` -eq 0 ]; then

             /etc/init.d/keepalived stop

    fi

fi

 

 

mysql -u root -ppassword -h 192.168.1.74

mysql> show databases;

+--------------------+

| Database           |

+--------------------+

| information_schema |

| galera_test7       |

| galera_test8       |

| mysql              |

| performance_schema |

| test               |

+--------------------+

6 rows in set (0.00 sec)

 

3.7. haproxy搭建及配置(两个节点都安装)及配置mariadb

yum install -y haproxy

vim /etc/haproxy/haproxy.cfg

listen galera_cluster

        mode tcp

        bind 0.0.0.0:3306

        balance roundrobin

        option tcpka

        option httpchk

       

        server maria-server1 192.168.1.72:3306 weight 1

        server maria-server2 192.168.1.73:3306 weight 1

3.8. 测试mysql集群是否正常

jumpserver1:

service keepalived start

service haproxy start

mysql -u root -pjumpserverDb -h 192.168.1.74

MariaDB [(none)]> show databases;

+--------------------+

| Database           |

+--------------------+

| galera_test7       |

| information_schema |

| mysql              |

| performance_schema |

+--------------------+

4 rows in set (0.00 sec)

 

jumpserver1:

service keepalived stop

 

mysql -u root -pjumpserverDb -h 192.168.1.74

MariaDB [(none)]> show databases;

+--------------------+

| Database           |

+--------------------+

| galera_test7       |

| information_schema |

| mysql              |

| performance_schema |

+--------------------+

4 rows in set (0.00 sec)

 

Jumpserver2:

3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000

    link/ether 08:00:27:4b:e1:39 brd ff:ff:ff:ff:ff:ff

    inet 192.168.1.76/23 brd 192.168.1.255 scope global eth1

inet 192.168.1.74/32 scope global eth1

 

说明Kvip已经浮动到jumpserver2 访问mysql正常。

 

 

4. Jumpserver安装

4.1 JMS_Master:

4.1.1.  数据库手动创建

mysql -e "create database jumpserver charset=‘utf8‘;"

mysql -e "grant all on jumpserver.* to ‘jumpserver‘@‘%‘ identified by ‘jumpserverDb‘;"

mysql -e "flush privileges;"

 

4.1.2.  安装git

 [root@JMS_Master ~]# yum -y install git

4.1.3  下载jumpserver,使用future-lz分支



[root@JMS_Master ~]# cd /opt
[root@JMS_Master opt]# git clone https://github.com/jumpserver/jumpserver.git -b future-lz

注:不要安装在/root、/home 等目录下,以免权限问题

 

4.1.4  执行安装脚本



[root@JMS_Master opt]# cd jumpserver/install
[root@JMS_Master install]# python install.py

安装到数据库这步选n

 

请输入您服务器的IP地址,用户浏览器可以访问 [192.168.1.75]: 192.168.1.75

是否安装新的MySQL服务器? (y/n) [y]: n

请输入数据库服务器IP [127.0.0.1]: 192.168.1.74

请输入数据库服务器端口 [3306]: 3306

请输入数据库服务器用户 [jumpserver]: jumpserver

请输入数据库服务器密码: jumpserverDb

请输入使用的数据库 [jumpserver]: jumpserver

连接数据库成功

请输入SMTP地址: xxx.qiye.163.com

请输入SMTP端口 [25]: 25

请输入账户: xxx@.com

请输入密码: xxx

 

        请登陆邮箱查收邮件, 然后确认是否继续安装

 

是否继续? (y/n) [y]: y

开始写入配置文件

开始安装Jumpserver ...

开始更新jumpserver

Creating tables ...

Creating table django_admin_log

Creating table auth_permission

Creating table auth_group_permissions

Creating table auth_group

Creating table django_content_type

Creating table django_session

Creating table setting

Creating table juser_usergroup

Creating table juser_user_group

Creating table juser_user_groups

Creating table juser_user_user_permissions

Creating table juser_user

Creating table juser_admingroup

Creating table juser_document

Creating table jasset_assetgroup

Creating table jasset_idc

Creating table jasset_asset_group

Creating table jasset_asset

Creating table jasset_assetrecord

Creating table jasset_assetalias

Creating table jperm_permlog

Creating table jperm_permsudo

Creating table jperm_permrole_sudo

Creating table jperm_permrole

Creating table jperm_permrule_asset_group

Creating table jperm_permrule_role

Creating table jperm_permrule_asset

Creating table jperm_permrule_user_group

Creating table jperm_permrule_user

Creating table jperm_permrule

Creating table jperm_permpush

Creating table jlog_log

Creating table jlog_alert

Creating table jlog_ttylog

Creating table jlog_execlog

Creating table jlog_filelog

Creating table jlog_termlog_user

Creating table jlog_termlog

Installing custom SQL ...

Installing indexes ...

Installed 0 object(s) from 0 fixture(s)

 

请输入管理员用户名 [admin]: jumpserverDb

请输入管理员密码: [jumpserverDb]: jumpserverDb

Starting jumpserver service:                               [  OK  ]

 

安装成功,请访问web, 祝你使用愉快。

请访问 https://github.com/jumpserver/jumpserver/wiki 查看文档

 

41.4  查看一下JMS的配置文件中key为90ga55qu7tgej8rr

[base]

url = http://192.168.1.75

key = 90ga55qu7tgej8rr

ip = 0.0.0.0

port = 8000

log = debug

 

[db]

engine = mysql

host = 192.168.1.74

port = 3306

user = jumpserver

password = jumpserverDb

database = jumpserver

 

[mail]

mail_enable = 1

email_host = xxx.qiye.163.com

email_port = 25

email_host_user = xxx@.com

email_host_password =xxx

email_use_tls = False

email_use_ssl = False

 

[connect]

nav_sort_by = ip

 

42 JMs_Slave:

42.1. 安装git

 
[root@JMS_Slave ~]# yum -y install git

4.2.2 下载jumpserver,使用future-lz分支

 

[root@JMS_Slave ~]# cd /opt
[root@JMS_Slave opt]# git clone https://github.com/jumpserver/jumpserver.git -b future-lz

注:不要安装在/root、/home 等目录下,以免权限问题

 

4.2.3 执行安装脚本

 

[root@JMS_Slave opt]# cd jumpserver/install/
[root@JMS_Slave install]# python install.py
 

  File "/usr/lib/python2.6/site-packages/pip/_vendor/requests/packages/urllib3/response.py", line 267, in read

    raise ReadTimeoutError(self._pool, None, ‘Read timed out.‘)

ReadTimeoutError: HTTPSConnectionPool(host=‘pypi.python.org‘, port=443): Read timed out.

 

安装JumpServer 依赖的python库失败!

 

解决:指定源安装

pip install -r requirements.txt -i http://pypi.douban.com/simple --trusted-host pypi.douban.com
[root@JMS_Slave install]# python install.py

安装到数据库这步选n

请输入您服务器的IP地址,用户浏览器可以访问 [192.168.1.76]: 192.168.1.76

是否安装新的MySQL服务器? (y/n) [y]: n

请输入数据库服务器IP [127.0.0.1]: 192.168.1.74

请输入数据库服务器端口 [3306]: 3306

请输入数据库服务器用户 [jumpserver]: jumpserver

请输入数据库服务器密码: jumpserverDb

请输入使用的数据库 [jumpserver]: jumpserver

连接数据库成功

请输入SMTP地址: smtp.qiye.163.com

请输入SMTP端口 [25]: 25

请输入账户: xxx@.com

请输入密码: xxx

 

        请登陆邮箱查收邮件, 然后确认是否继续安装

 

是否继续? (y/n) [y]: y

开始写入配置文件

开始安装Jumpserver ...

开始更新jumpserver

Creating tables ...

Installing custom SQL ...

Installing indexes ...

Installed 0 object(s) from 0 fixture(s)

 

请输入管理员用户名 [admin]: jumpserverDb

请输入管理员密码: [5Lov@wife]: jumpserverDb

请再次输入管理员密码: [5Lov@wife]: jumpserverDb

Starting jumpserver service:                               [  OK  ]

 

安装成功,请访问web, 祝你使用愉快。

请访问 https://github.com/jumpserver/jumpserver/wiki 查看文档

 

4.2.4 key = nf85skm5dh9oenr9改成 90ga55qu7tgej8rr 要和JMS_Master的key值一样


复制代码

[root@JMS_Slave install]# cd ..
[root@JMS_Slave jumpserver]# vim jumpserver.conf 

[base]

url = http://192.168.1.76

key = 90ga55qu7tgej8rr

ip = 0.0.0.0

port = 8000

log = debug

 

[db]

engine = mysql

host = 192.168.1.74

port = 3306

user = jumpserver

password = jumpserverDb

database = jumpserver

 

[mail]

mail_enable = 1

email_host = smtp.qiye.163.com

email_port = 25

email_host_user = xxx@.com

email_host_password = xxx

email_use_tls = False

email_use_ssl = False

 

[connect]

nav_sort_by = ip

4.2.5 重启一下服务让修改配置生效


[root@JMS_Slave jumpserver]# ./service.sh restart
Stopping jumpserver service:                               [  OK  ]
Starting jumpserver service:                               [  OK  ]

访问web并登录一下

 

 两台JMS之间做密钥免密码登录:

参照:http://www.2cto.com/os/201205/133514.html

5. 两台JMS之间key&user同步

参考:http://bbs.jumpserver.org/read/617.html

 

 

6. Haproxy配置Jumpserver

#---------------------------------------------------------------------

# jumpserver 8080

#---------------------------------------------------------------------

frontend jumpserver

        bind    *:8080

        mode    http

        option  httplog

        option  httpclose

        option  forwardfor

        log     global

        use_backend     jumpserver_backend

 

backend jumpserver_backend

        balance roundrobin

        cookie  ServerId insert indirect nocache

        server  jumpserver_1 192.168.1.75:8000 cookie A check

        server  jumpserver_2 192.168.1.76:8000 cookie B check backup

 

192.168.1.76为备份服务当主服务出现问题后自动接管。

 


 [i1]确保内网中router_id没有出现88

 [i2]优先级高

jumpserver篇--安装(高可用性 mariadb+haproxy)