首页 > 代码库 > LINUX 软RAID和硬RAI及软RAID的实现
LINUX 软RAID和硬RAI及软RAID的实现
一、RAID简介
RAID是“Redundant Array of Independent Disk”的缩写，中文意思是独立冗余磁盘阵列，早期称为廉价磁盘阵列。简单地解释，就是将N台硬盘通过RAID Controller（分Hardware，Software）结合成虚拟单台大容量的硬盘使用。RAID技术大致分为两种：基于硬件的RAID技术和基于软件的RAID技术。
二、RAID术语
RAID0
raid0: 读、写性能提升，无容错能力，空间n*disk
RAID1
raid1: 写性能略有下降，读性能提升，容错，空间：1*disk
RAID4
raid4: 读、写性能提升，有容错能力（最多坏一块磁盘），空间：(n-1)*disk
RAID5
raid5: 读、写性能提升，有容错能力（最多坏一块磁盘），空间：(n-1)*disk
RAID10
raid10: 读、写性能提升，有容错能力（每一组可坏一块盘），空间：n*disk/2
三、RAID的基本命令
mdadm工具：
md: 支持将任何块设备组织成RAID
mdadm: 模块式化命令
-A: 装配模式，重新识别此前实现的RAID
-C：创建模式，创建RAID
-F：监控模式
管理模式：-f, -r, -a
-C: 创建模式中专用选项
-n #: 用于创建RAID设备的磁盘个数；
-l #: 级别
-a yes: 自动为创建的RAID生成设备文件;
-c Chunk_Size:
md设备的设备文件，一般在/dev目录下，以md开头，后跟一个数字来区别
四、 安装RAID软件
首先确定系统中是否安装了工具,没有安装的话要安装之后才能使用，否则没有命令功能支持，我的是centos6.5默认安装了。如果输入命令没有提示没有查询到就要安装raid程序啦。
[root@dingchao tmp]# rpm -qa mdadm 查询是否安装了支持raid的软件
mdadm-3.2.6-7.el6.x86_64
五、实验
实验是划分四个5g的硬盘，演示raid的创建、热备盘、停止、添加、重新装配
1、添加磁盘用虚拟机添加一个20g的硬盘，此处省略
2、创建分区
[root@localhost ~]# fdisk /dev/sdb
Command (m for help): n 创建主分区
Command action
e extended 逻辑分区
p primary partition (1-4)
p
Partition number (1-4): 1 第一个分区
First cylinder (1-2610, default 1): 开始块
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-2610, default 2610): +5G 划分5G容量
Command (m for help): t 修改文件类型
Selected partition 1
Hex code (type L to list codes): 文件类型(Linux raid autodetect)
Changed system type of partition 1 to 8e ( Linux raid autodetect)
Command (m for help): p 查看分区
Disk /dev/sdb: 21.5 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xba762e8d
Device Boot Start End Blocks Id System
/dev/sdb1 1 654 5253223+ fd Linux raid autodetect
/dev/sdb2 655 1308 5253255 fd Linux raid autodetect
/dev/sdb3 1309 1962 5253255 fd Linux raid autodetect
Command (m for help): w 保存分区
The partition table has been altered!
创建分区此处不再演示，照着上面的重做分好实验要求的分区
3、查看所创建分区系统是否识别到、没有识别到要 partx -a /dev/sdbx直到识别到或者重启系统。
[root@localhost ~]# cat /proc/partitions 查看创建分区
8 0 83886080 sda
8 1 204800 sda1
8 2 62914560 sda2
8 3 39163 sda3
8 4 1 sda4
8 5 40131 sda5
8 16 20971520 sdb
8 17 5253223 sdb1
8 18 5253255 sdb2
8 19 5253255 sdb3
4、创建 RAID MD5
[root@dingchao ~]# mdadm -C /dev/md5 -a yes -n 3 -l 5 /dev/sdb1 /dev/sdb2 /dev/sdb3
mdadm: Defaulting to version 1.2 metadata 创建一个名叫MD5的raid自动创建、三块磁盘、级别
mdadm: array /dev/md5 started. 为五 添加的磁盘是/dev/sdb1 /dev/sdb2 /dev/sdb3
mdadm: array /dev/md5 started.
[root@dingchao ~]# cat /proc/mdstat 查看创建信息
Personalities : [raid6] [raid5] [raid4]
md5 : active raid5 sdb3[3] sdb2[1] sdb1[0]
10498048 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/2] [UU_]
[=>...................] recovery = 5.9% (311296/5249024) finish=3.6min speed=22235K/sec
unused devices: <none>
5、格式化挂载
[root@dingchao ~]# mke2fs -t ext4 /dev/md5 格式化
mke2fs 1.41.12 (17-May-2010)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=128 blocks, Stripe width=256 blocks
657072 inodes, 2624512 blocks
131225 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=2688548864
81 block groups
32768 blocks per group, 32768 fragments per group
8112 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 23 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
[root@dingchao ~]# mkdir /tmp/user1 创建目录
[root@dingchao ~]# mount /dev/md5 //tmp/user1 挂载
[root@dingchao ~]# mount 查看挂载信息
/dev/mapper/vg0-root on / type ext4 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
tmpfs on /dev/shm type tmpfs (rw)
/dev/sda1 on /boot type ext4 (rw)
/dev/mapper/vg0-usr on /usr type ext4 (rw)
/dev/mapper/vg0-var on /var type ext4 (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
/dev/md5 on /tmp/user1 type ext4 (rw)
[root@dingchao ~]# df -lh 查看 RAID md5 信息
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg0-root 20G 406M 19G 3% /
tmpfs 935M 0 935M 0% /dev/shm
/dev/sda1 194M 35M 150M 19% /boot
/dev/mapper/vg0-usr 9.9G 2.0G 7.4G 21% /usr
/dev/mapper/vg0-var 20G 305M 19G 2% /var
/dev/md5 9.9G 151M 9.3G 2% /tmp/user1
[root@dingchao ~]# df -lh /dev/md5
Filesystem Size Used Avail Use% Mounted on
/dev/md5 9.9G 151M 9.3G 2% /tmp/user1
[root@dingchao ~]# mdadm --detail /dev/md5 查看 RAID md5 的详细信息
/dev/md5:
Version : 1.2
Creation Time : Sat Jul 12 06:53:18 2014
Raid Level : raid5 级别
Array Size : 10498048 (10.01 GiB 10.75 GB) 容量
Used Dev Size : 5249024 (5.01 GiB 5.38 GB)
Raid Devices : 3 几块
Total Devices : 3
Persistence : Superblock is persistent
Update Time : Sat Jul 12 06:59:10 2014
State : clean
Active Devices : 3 活动三块
Working Devices : 3 工作三块
Failed Devices : 0 停止工作0块
Spare Devices : 0 热备0块
Layout : left-symmetric
Chunk Size : 512K
Name : dingchao.localdomain:5 (local to host dingchao.localdomain)
UUID : 5e20abb1:5dddca66:b82c6498:32052563
Events : 24
Number Major Minor RaidDevice State
0 8 17 0 active sync /dev/sdb1
1 8 18 1 active sync /dev/sdb2
3 8 19 2 active sync /dev/sdb3 那几块磁盘在工作
6、新增一块做热备及演示磁盘坏盘，添加，移除
重新添加一块磁盘分区此处不做演示、上面照做
[root@dingchao ~]# kpartx -af /dev/sda6 添加sda6这块硬盘
[root@dingchao ~]# mdadm /dev/md5 -f /dev/sdb1 模拟sdb1 坏掉
mdadm: set /dev/sdb1 faulty in /dev/md5
[root@dingchao ~]# mdadm --detail /dev/md5 查看 RAID md5 的详细信息
/dev/md5:
Version : 1.2
Creation Time : Sat Jul 12 06:53:18 2014
Raid Level : raid5
Array Size : 10498048 (10.01 GiB 10.75 GB)
Used Dev Size : 5249024 (5.01 GiB 5.38 GB)
Raid Devices : 3
Total Devices : 3
Persistence : Superblock is persistent
Update Time : Sat Jul 12 07:23:03 2014
State : clean, degraded
Active Devices : 2
Working Devices : 2
Failed Devices : 1 坏一块磁盘的表现
Spare Devices : 0
Layout : left-symmetric
Chunk Size : 512K
Name : dingchao.localdomain:5 (local to host dingchao.localdomain)
UUID : 5e20abb1:5dddca66:b82c6498:32052563
Events : 26
Number Major Minor RaidDevice State
0 0 0 0 removed 提示 removed
1 8 18 1 active sync /dev/sdb2
3 8 19 2 active sync /dev/sdb3
0 8 17 - faulty /dev/sdb1 提示sdb1坏掉
[root@dingchao ~]# mdadm /dev/md5 -r /dev/sdb1 移除坏掉的sdb1
mdadm: hot removed /dev/sdb1 from /dev/md5
[root@dingchao ~]# mdadm --detail /dev/md5 查看 RAID md5 的详细信息
/dev/md5:
Version : 1.2
Creation Time : Sat Jul 12 06:53:18 2014
Raid Level : raid5
Array Size : 10498048 (10.01 GiB 10.75 GB)
Used Dev Size : 5249024 (5.01 GiB 5.38 GB)
Raid Devices : 3
Total Devices : 2
Persistence : Superblock is persistent
Update Time : Sat Jul 12 07:23:24 2014
State : clean, degraded
Active Devices : 2
Working Devices : 2 状态信息
Failed Devices : 0
Spare Devices : 0
Layout : left-symmetric
Chunk Size : 512K
Name : dingchao.localdomain:5 (local to host dingchao.localdomain)
UUID : 5e20abb1:5dddca66:b82c6498:32052563
Events : 29
Number Major Minor RaidDevice State
0 0 0 0 removed 坏盘已被移除
1 8 18 1 active sync /dev/sdb2
3 8 19 2 active sync /dev/sdb3
[root@dingchao ~]# mdadm /dev/md5 -a /dev/sda6 添加一块已经分好区的硬盘带RAID MD5中
mdadm: added /dev/sda6
[root@dingchao ~]# mdadm --detail /dev/md5 查看 RAID md5 的详细信息
/dev/md5:
Version : 1.2
Creation Time : Sat Jul 12 06:53:18 2014
Raid Level : raid5
Array Size : 10498048 (10.01 GiB 10.75 GB)
Used Dev Size : 5249024 (5.01 GiB 5.38 GB)
Raid Devices : 3
Total Devices : 3 总的信息
Persistence : Superblock is persistent
Update Time : Sat Jul 12 07:29:33 2014
State : clean, degraded, recovering
Active Devices : 2
Working Devices : 3 现在的信息
Failed Devices : 0
Spare Devices : 1 热备信息
Layout : left-symmetric
Chunk Size : 512K
Rebuild Status : 21% complete 重建信息
Name : dingchao.localdomain:5 (local to host dingchao.localdomain)
UUID : 5e20abb1:5dddca66:b82c6498:32052563
Events : 38
Number Major Minor RaidDevice State
4 8 6 0 spare rebuilding /dev/sda6 热备重建
1 8 18 1 active sync /dev/sdb2
3 8 19 2 active sync /dev/sdb3
[root@dingchao ~]# mdadm /dev/md5 -a /dev/sdb1 添加sdb1作为热备盘
mdadm: added /dev/sdb1
[root@dingchao ~]# mdadm --detail /dev/md5 查看 RAID md5 的详细信息
/dev/md5:
Version : 1.2
Creation Time : Sat Jul 12 06:53:18 2014
Raid Level : raid5
Array Size : 10498048 (10.01 GiB 10.75 GB)
Used Dev Size : 5249024 (5.01 GiB 5.38 GB)
Raid Devices : 3
Total Devices : 4 总的信息
Persistence : Superblock is persistent
Update Time : Sat Jul 12 07:29:57 2014
State : clean, degraded, recovering
Active Devices : 2
Working Devices : 4 现在的信息
Failed Devices : 0
Spare Devices : 2 热备信息
Layout : left-symmetric
Chunk Size : 512K
Rebuild Status : 39% complete 重建信息
Name : dingchao.localdomain:5 (local to host dingchao.localdomain)
UUID : 5e20abb1:5dddca66:b82c6498:32052563
Events : 44
Number Major Minor RaidDevice State
4 8 6 0 spare rebuilding /dev/sda6 热备重建
1 8 18 1 active sync /dev/sdb2
3 8 19 2 active sync /dev/sdb3
5 8 17 - spare /dev/sdb1 空闲热备盘
7、多一块热备、盘坏了自动添加重建
[root@dingchao ~]# mdadm /dev/md5 -f /dev/sda6 模拟磁盘sdb6坏掉
mdadm: set /dev/sda6 faulty in /dev/md5
[root@dingchao ~]# mdadm /dev/md5 -r /dev/sda6 移除sdb6
mdadm: hot remove failed for /dev/sda6: Device or resource busy
[root@dingchao ~]# mdadm --detail /dev/md5 查看 RAID md5 的详细信息
/dev/md5:
Version : 1.2
Creation Time : Sat Jul 12 06:53:18 2014
Raid Level : raid5
Array Size : 10498048 (10.01 GiB 10.75 GB)
Used Dev Size : 5249024 (5.01 GiB 5.38 GB)
Raid Devices : 3
Total Devices : 4 总盘的信息
Persistence : Superblock is persistent
Update Time : Sat Jul 12 07:42:30 2014
State : clean, degraded, recovering
Active Devices : 2
Working Devices : 3
Failed Devices : 1
Spare Devices : 1 工作信息
Layout : left-symmetric
Chunk Size : 512K
Rebuild Status : 5% complete
Name : dingchao.localdomain:5 (local to host dingchao.localdomain)
UUID : 5e20abb1:5dddca66:b82c6498:32052563
Events : 59
Number Major Minor RaidDevice State
5 8 17 0 spare rebuilding /dev/sdb1
sdb1热备自动顶上重建
1 8 18 1 active sync /dev/sdb2
3 8 19 2 active sync /dev/sdb3
4 8 6 - faulty /dev/sda6 提示sdb6磁盘坏掉
8、模拟停电、机器坏掉之后、停盘、重新装载
[root@dingchao ~]# umount /tmp/user1 卸载
[root@dingchao ~]# mount 挂载信息
/dev/mapper/vg0-root on / type ext4 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
tmpfs on /dev/shm type tmpfs (rw)
/dev/sda1 on /boot type ext4 (rw)
/dev/mapper/vg0-usr on /usr type ext4 (rw)
/dev/mapper/vg0-var on /var type ext4 (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
[root@dingchao ~]# mdadm -S /dev/md5 停止 RAID MD5
mdadm: stopped /dev/md5
[root@dingchao tmp]# mdadm -A /dev/md5 /dev/sdb{1,2,3} /dev/sda6 重新装载MD5
mdadm: /dev/md5 has been started with 3 drives and 1 spare.
[root@dingchao tmp]# mdadm --detail /dev/md5 查看 RAID md5 的详细信息
/dev/md5:
Version : 1.2
Creation Time : Sat Jul 12 06:53:18 2014
Raid Level : raid5
Array Size : 10498048 (10.01 GiB 10.75 GB)
Used Dev Size : 5249024 (5.01 GiB 5.38 GB)
Raid Devices : 3
Total Devices : 4 总盘的信息
Persistence : Superblock is persistent
Update Time : Sat Jul 12 07:52:33 2014
State : clean
Active Devices : 3
Working Devices : 4
Failed Devices : 0
Spare Devices : 1 状态信息
Layout : left-symmetric
Chunk Size : 512K
Name : dingchao.localdomain:5 (local to host dingchao.localdomain)
UUID : 5e20abb1:5dddca66:b82c6498:32052563
Events : 84
Number Major Minor RaidDevice State
5 8 17 0 active sync /dev/sdb1
1 8 18 1 active sync /dev/sdb2
3 8 19 2 active sync /dev/sdb3
4 8 6 - spare /dev/sda6 磁盘工作状态
[root@dingchao tmp]# mount /dev/md5 /tmp/user1 重新挂载就可以使用啦
[root@dingchao tmp]# mount 查看挂载信息
/dev/mapper/vg0-root on / type ext4 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
tmpfs on /dev/shm type tmpfs (rw)
/dev/sda1 on /boot type ext4 (rw)
/dev/mapper/vg0-usr on /usr type ext4 (rw)
/dev/mapper/vg0-var on /var type ext4 (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
/dev/md5 on /tmp/user1 type ext4 (rw) 查看挂载信息
六、总结
RAID: 组织多块硬盘当一个设备来使用
硬件RAID：
硬件控制器：创建RAID通过BIOS进行
在OS中看到的仅是一个单独的设备
软件RAID：
无需任何硬件的RAID设备，仅需多个块设备（磁盘分区即可）
在OS中看到的是多个基本的磁盘设备、磁盘分区等，而后将这多个块设备可以组织一个单独的设备使用
即为软RAID
装配模式：在某OS上创建的软件RAID，被迁移到其它主机上，并启动OS之后，Linux auto detect
本文出自 “天涯海角” 博客，谢绝转载！