首页 > 代码库 > Glusterfs 分布式存储安装部署

Glusterfs 分布式存储安装部署

2024-08-15 22:32:45   222人阅读

 

Glusterfs 分布式存储部署 是存储当中可以选择的一种 现在很多虚拟化 云计算都在用软件存储 例如 ceph

Glusterfs 等等 今天我们部署一下Glusterfs环境

 

GlusterFs安装

 

环境信息：

         Glusterfs-A                          10.20.0.200                              
         Glusterfs-B                  10.20.0.201
         Glusterfs-C                  10.20.0.202

 

 

 

服务器版本信息

# cat /etc/redhat-release CentOS release 6.6 (Final)

  # uname -r
  2.6.32-504.el6.x86_64

 

软件下载地址

http://download.gluster.org/pub/gluster/glusterfs/3.4/3.4.6/CentOS/epel-6.6/x86_64

 

依赖包安装

直接用Yum安装 每台都需要安装

# yum -y install libibverbs librdmacm xfsprogs nfs-utils rpcbind  libaio liblvm2app lvm2-devel

 

软件安装

安装服务器端软件包

glusterfs-3.4.6-1.el6.x86_64.rpmglusterfs-api-3.4.6-1.el6.x86_64.rpmglusterfs-cli-3.4.6-1.el6.x86_64.rpmglusterfs-fuse-3.4.6-1.el6.x86_64.rpmglusterfs-libs-3.4.6-1.el6.x86_64.rpmglusterfs-server-3.4.6-1.el6.x86_64.rpm

安装客户端软件包

glusterfs-3.4.6-1.el6.x86_64.rpmglusterfs-fuse-3.4.6-1.el6.x86_64.rpmglusterfs-libs-3.4.6-1.el6.x86_64.rpm

 

磁盘配置

磁盘分区：

EXT4格式化工具目前最大支持16TB

使用parted进行磁盘分区

分区格式化
每个服务器上面都需要至少添加一块硬盘 我们这边是用的1块8G硬盘 现在每台都格式化

# mkfs.ext4 -L /brick1 /dev/sdbmke2fs 1.41.12 (17-May-2010)/dev/sdb is entire device, not just one partition!Proceed anyway? (y,n) yFilesystem label=/brick1OS type: LinuxBlock size=4096 (log=2)Fragment size=4096 (log=2)Stride=0 blocks, Stripe width=0 blocks524288 inodes, 2097152 blocks104857 blocks (5.00%) reserved for the super userFirst data block=0Maximum filesystem blocks=214748364864 block groups32768 blocks per group, 32768 fragments per group8192 inodes per groupSuperblock backups stored on blocks:     32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632Writing inode tables: done                            Creating journal (32768 blocks): doneWriting superblocks and filesystem accounting information: doneThis filesystem will be automatically checked every 39 mounts or180 days, whichever comes first.  Use tune2fs -c or -i to override.

 

自动挂载分区

# vim /etc/fstab LABEL=/brick1  /brick1 ext4  defaults 0 0

 

mount文件系统

# mkdir /brick1# mount -L /brick1 /brick1/# df -khFilesystem            Size  Used Avail Use% Mounted on/dev/mapper/VolGroup-lv_root                       47G  759M   44G   2% /tmpfs                 491M     0  491M   0% /dev/shm/dev/sda1             477M   28M  425M   6% /boot/dev/sdb              7.8G   18M  7.4G   1% /brick1

 

基础测试包安装 性能测试包

工具软件
Atop,iperf,sysstat

dd,lozone,fio,postmark

工具安装

#rpm -ivh http://mirrors.ustc.edu.cn/fedora/epel/6/x86_64/epel-release-6-8.noarch.rpm

yum install sysstat

Gcc -o postmark postmark-1.52.c

有些可以用yum安装

 

安装GlusterFS软件包

每台创建存放软件的目录  我们目前安装的是3.4.6的版本

#mdkir /home/tools/3.4.6 -p
#cd /home/tools/3.4.6# lltotal 2164-rw-r--r-- 1 root root 1027320 Oct  4 11:13 glusterfs-3.4.6-1.el6.x86_64.rpm-rw-r--r-- 1 root root   66680 Oct  4 11:13 glusterfs-api-3.4.6-1.el6.x86_64.rpm-rw-r--r-- 1 root root  108872 Oct  4 11:14 glusterfs-cli-3.4.6-1.el6.x86_64.rpm-rw-r--r-- 1 root root   88452 Oct  4 11:14 glusterfs-fuse-3.4.6-1.el6.x86_64.rpm-rw-r--r-- 1 root root  124288 Oct  4 11:14 glusterfs-geo-replication-3.4.6-1.el6.x86_64.rpm-rw-r--r-- 1 root root  227900 Oct  4 11:14 glusterfs-libs-3.4.6-1.el6.x86_64.rpm-rw-r--r-- 1 root root   50128 Oct  4 11:14 glusterfs-rdma-3.4.6-1.el6.x86_64.rpm-rw-r--r-- 1 root root  504744 Oct  4 11:14 glusterfs-server-3.4.6-1.el6.x86_64.rpm

直接rpm安装

# rpm -ivh *.rpmwarning: glusterfs-3.4.6-1.el6.x86_64.rpm: Header V4 RSA/SHA1 Signature, key ID 4ab22bb3: NOKEYPreparing...                ########################################### [100%]   1:glusterfs-libs         ########################################### [ 13%]   2:glusterfs              ########################################### [ 25%]   3:glusterfs-fuse         ########################################### [ 38%]   4:glusterfs-cli          ########################################### [ 50%]   5:glusterfs-server       ########################################### [ 63%]   6:glusterfs-geo-replicati########################################### [ 75%]   7:glusterfs-api          ########################################### [ 88%]   8:glusterfs-rdma         ########################################### [100%]

 

安装成功启动GlusterFS

# service glusterd startStarting glusterd:                                         [  OK  ]# chkconfig glusterd on

 

组建集群扩展

在GlusterFS-01服务器上面操作 选择10.20.0.200 作为当前集群节点

添加另外两台服务器

# gluster peer probe 10.20.0.201peer probe: success# gluster peer probe 10.20.0.202peer probe: success

 

GlusterFS-01服务器查看添加状态

# gluster peer statusNumber of Peers: 2Hostname: 10.20.0.201Port: 24007Uuid: c4d9edfd-d6ae-4628-8ab3-5a9fbb5f18fcState: Peer in Cluster (Connected)Hostname: 10.20.0.202Port: 24007Uuid: c0b425bb-a7b4-4dae-bc19-ea6031bf02b9State: Peer in Cluster (Connected)

GlusterFS-02服务器查看添加状态

# gluster peer statusNumber of Peers: 2Hostname: 10.20.0.200Port: 24007Uuid: 7ad4db05-ee75-44e6-8213-e58072b62d33State: Peer in Cluster (Connected)Hostname: 10.20.0.202Uuid: c0b425bb-a7b4-4dae-bc19-ea6031bf02b9State: Peer in Cluster (Connected)

 

GlusterFS-03服务器查看添加状态

# gluster peer statusNumber of Peers: 2Hostname: 10.20.0.200Port: 24007Uuid: 7ad4db05-ee75-44e6-8213-e58072b62d33State: Peer in Cluster (Connected)Hostname: 10.20.0.201Uuid: c4d9edfd-d6ae-4628-8ab3-5a9fbb5f18fcState: Peer in Cluster (Connected)

表示成功

 

如果添加不成功 排除错误：

1，每台服务器是否能否Ping通

2，防火墙是否关闭iptables selinux

3，glusterd服务是否启动成功

 

创建卷

  • 确定创建卷的类型
  • 确定创建卷的brick列表
  • 确实创建卷的网络类型（TCP/RDMA）

Gluster volume create创建卷

在GlusterFS-01服务器上面操作 可以任选一台

# gluster volume create dht-vol1 10.20.0.200:/brick1/b1 10.20.0.201:/brick1/b2 10.20.0.202:/brick1/b3volume create: dht-vol1: success: please start the volume to access data

 

查看卷状态信息

# gluster volume info Volume Name: dht-vol1Type: DistributeVolume ID: 7d947cf1-c743-477e-a28d-7d030572b655Status: CreatedNumber of Bricks: 3Transport-type: tcpBricks:Brick1: 10.20.0.200:/brick1/b1Brick2: 10.20.0.201:/brick1/b2Brick3: 10.20.0.202:/brick1/b3

启动卷

# gluster volume start dht-vol1volume start: dht-vol1: success

 

Glusterfs 分布式存储安装部署


联系
我们