首页 > 代码库 > Cent OS 6上实现IP SAN存储
Cent OS 6上实现IP SAN存储
一、前提介绍
1、存储类型介绍
DAS:直接附加存储,是最为常见的存储方式,日常所见的笔记本,U盘等都是这种方式。其主要支持的硬盘类型有并行接口的IDE,SCSI;串行接口的SATA,SAS,USB等。
NAS:是由文件服务协议实现的文件服务器,其主要表现为NFS,CIFS。
SAN:存储区域网络,可利用现有的成熟网络技术承载存储协议SCSI的相关报文。
2、SCSI介绍
SCSI是一种I/O技术,规范了一种并行的I/O总线的相关的协议,它的数据传输是以块的方式进行的,其具有设备无关性(target管理的lun下的存储设备是什么接口都行,只要能提供存储)、多设备并行(每个通道连接多个target,每个target管理多个lun)、高带宽(SCSI协议分层,将物理层的传输介质换为万兆以太网或光纤,提供非常高的带宽)、低系统开销(将由内核完成的转换iSCSI协议报文交由nic实现)的特点,并采取了协议分层的功能,分为三层,应用层、传输层、物理层,从而可实现多种存储解决方案。
3、实验介绍
iSCSI server: 192.168.0.20(node3)
Initiator 1:192.168.0.15(node1)
Initiator 1:192.168.0.16(node2)
关闭iptables或定义规则放行3260端口
关闭selinux
二、配置 iSCSI server
1、准备磁盘设备
[root@node3 ~]# fdisk -l Disk /dev/sdb: 10.7 GB, 10737418240 bytes 255 heads, 63 sectors/track, 1305 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 Disk /dev/sdc: 10.7 GB, 10737418240 bytes 255 heads, 63 sectors/track, 1305 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000
2、安装target包,并启动服务
[root@node3 ~]# yum install scsi-target-utils -y [root@node3 ~]# service tgtd start Starting SCSI target daemon: [ OK ] [root@node3 ~]# chkconfig tgtd on
3、创建target,创建lun,并授权
3.1、使用tgtadm配置,重启后失效
tgtadm:全功能的命令行配置工具,其配置结果在OS重启后会失效 tgtadm命令:-h 获取帮助信息 模式化工具操作: tgtadm:管理target logicalunit:管理lun account:用户帐号 管理操作: show:查看 new:新建 delete:删除 update:修改 bind:绑定,即实现授权,通过将IP或账号与target绑定进行 unbind:解除授权 常用选项: --lld, -L <driver> 指明驱动,iscsi --targetname,-T <targetname> 指明target名字,使用iqn格式的 --control-port,-C <port> 指明监听端口 --bstype,-E <type> 指明磁盘类型 --mode, -m <mode> 指明类型,target或lun --op,-o <operation> 进行什么操作 --tid, -t <TID> 指明tid --lun, -l <LUN> 指明lun编号,0为控制器,从1开始 --backing-store,-b <PATH> 指明lun连接的设备 --initiator-address,-I <address> 指明授权的网段 --initiator-name,-Q <name> 指明授权的用户名 --name,-n <parameter> --value,-V <value> --params,-P <param=value[,param=value...]> --force, -F 强制操作 --help, -h 帮助信息 #创建target [root@node3 ~]# tgtadm --lld iscsi --mode target --op new --tid 1 --targetname iqn.2016-11.com.xj:s1.t1 #创建lun [root@node3 ~]# tgtadm --lld iscsi --mode logicalunit --op new --tid 1 --lun 1 --backing-store /dev/sdb [root@node3 ~]# tgtadm --lld iscsi --mode logicalunit --op new --tid 1 --lun 2 --backing-store /dev/sdc #授权(基于ip)与基于用户的二选一,一般使用基于IP的 [root@node3 ~]# tgtadm --lld iscsi --mode target --op bind --tid 1 --initiator-address 192.168.0.0/24 #授权(基于用户),并绑定target [root@node3 ~]# tgtadm -L iscsi -m account -o new --user jym --password jym [root@node3 ~]# tgtadm -L iscsi -m account -o bind --tid 1 --user jym #查看创建的target和lun [root@node3 ~]# tgtadm --lld iscsi --mode target --op show Target 1: iqn.2016-11.com.xj:s1.t1 System information: Driver: iscsi State: ready I_T nexus information: LUN information: LUN: 0 Type: controller SCSI ID: IET 00010000 SCSI SN: beaf10 Size: 0 MB, Block size: 1 Online: Yes Removable media: No Prevent removal: No Readonly: No Backing store type: null Backing store path: None Backing store flags: LUN: 1 Type: disk SCSI ID: IET 00010001 SCSI SN: beaf11 Size: 10737 MB, Block size: 512 Online: Yes Removable media: No Prevent removal: No Readonly: No Backing store type: rdwr Backing store path: /dev/sdb Backing store flags: LUN: 2 Type: disk SCSI ID: IET 00010002 SCSI SN: beaf12 Size: 10737 MB, Block size: 512 Online: Yes Removable media: No Prevent removal: No Readonly: No Backing store type: rdwr Backing store path: /dev/sdc Backing store flags: Account information: ACL information: 192.168.0.0/24
3.2、使用tgt-admin配置
通过读取配置文件/etc/tgt/target.conf进行配置的工具 [root@node3 ~]# cp /etc/tgt/targets.conf /etc/tgt/targets.conf.bak [root@node3 ~]# vim /etc/tgt/targets.conf <target iqn.2016-11.com.xj:s2.t1> backing-store /dev/sdb backing-store /dev/sdc initiator-address 192.168.0.0/24 #基于ip # incominguser user1 secretpass12 #基于用户 # outgoinguser userA secretpassA </target>
4、配置 iSCSI initiator(node1,node2同步)
1、安装程序包,配置initiator的名字,并启动服务
#安装程序包 [root@node1 ~]# yum -y install iscsi-initiator-utils #使用iscsi-iname生成initiator的名字 [root@node1 ~]# echo "InitiatorName=$(iscsi-iname -p iqn.2016-11.com.xj)" > /etc/iscsi/initiatorname.iscsi [root@node1 ~]# cat /etc/iscsi/initiatorname.iscsi InitiatorName=iqn.2016-11.com.xj:3f11231da124 #启动iscsi服务与守护进程 [root@node1 ~]# service iscsi start [root@node1 ~]# service iscsid start
2、使用iscsiadm实现target的发现、注册等
(1)发现模式:discovery
iscsiadm命令 常用选项: -m:指明模式 -h:帮助信息 -v:详细信息 -d:调试等级 -p:显示级别 -t:类型 -p: ip:port 在ip的port上发现 -l:登录进去 基于用户名(填入对应的用户名密码) [root@node1 ~]# vim /etc/iscsi/iscsid.conf node.session.auth.username = username node.session.auth.password = password discovery.sendtargets.auth.username = username discovery.sendtargets.auth.password = password 基于ip [root@node1 ~]# iscsiadm -m discovery -d 3 -t st -p 192.168.0.20:3260 iscsiadm: ip 192.168.0.20, port 3260, tgpt -1 iscsiadm: Max file limits 1024 4096 iscsiadm: starting sendtargets discovery, address 192.168.0.20:3260, iscsiadm: connecting to 192.168.0.20:3260 iscsiadm: connected local port 41839 to 192.168.0.20:3260 iscsiadm: connected to discovery address 192.168.0.20 iscsiadm: login response status 0000 iscsiadm: discovery process to 192.168.0.20:3260 exiting iscsiadm: disconnecting conn 0x9fba78, fd 3 192.168.0.20:3260,1 iqn.2016-11.com.xj:s1.t1 验证发现信息 [root@node1 ~]# ls /var/lib/iscsi/send_targets/192.168.0.20,3260/ iqn.2016-11.com.xj:s1.t1,192.168.0.20,3260,1,default st_config
(2)节点模式:node
# -l:登入,-u:登出,-o delete:删除条目 [root@node1 ~]# iscsiadm -m node -d 1 -T iqn.2016-11.com.xj:s1.t1 -p 192.168.0.20:3260 -l Logging in to [iface: default, target: iqn.2016-11.com.xj:s1.t1, portal: 192.168.0.20,3260] (multiple) Login to [iface: default, target: iqn.2016-11.com.xj:s1.t1, portal: 192.168.0.20,3260] successful.
验证登录节点(在node1上就可以看到sdb,sdc了)
[root@node1 ~]# fdisk -l /dev/sd[a-z] Disk /dev/sda: 21.5 GB, 21474836480 bytes 255 heads, 63 sectors/track, 2610 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00058545 Device Boot Start End Blocks Id System /dev/sda1 * 1 64 512000 83 Linux Partition 1 does not end on cylinder boundary. /dev/sda2 64 2611 20458496 8e Linux LVM Disk /dev/sdb: 10.7 GB, 10737418240 bytes 64 heads, 32 sectors/track, 10240 cylinders Units = cylinders of 2048 * 512 = 1048576 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 Disk /dev/sdc: 10.7 GB, 10737418240 bytes 64 heads, 32 sectors/track, 10240 cylinders Units = cylinders of 2048 * 512 = 1048576 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000
5、验证iscsi
1、在initiator 1上创建并格式化分区,并挂载
分区 [root@node1 ~]# fdisk /dev/sdb Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel Building a new DOS disklabel with disk identifier 0x6202406e. Changes will remain in memory only, until you decide to write them. After that, of course, the previous content won‘t be recoverable. Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite) WARNING: DOS-compatible mode is deprecated. It‘s strongly recommended to switch off the mode (command ‘c‘) and change display units to sectors (command ‘u‘). Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-10240, default 1): Using default value 1 Last cylinder, +cylinders or +size{K,M,G} (1-10240, default 10240): +5G Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks. 格式化 [root@node1 ~]# mke2fs -t ext4 /dev/sdb1 挂载测试 [root@node1 ~]# mount /dev/sdb1 /mnt [root@node1 ~]# cd /mnt [root@node1 mnt]# ll total 16 drwx------. 2 root root 16384 Nov 25 21:53 lost+found [root@node1 mnt]# touch text.txt [root@node1 mnt]# ll total 16 drwx------. 2 root root 16384 Nov 25 21:53 lost+found -rw-r--r--. 1 root root 0 Nov 25 21:57 text.txt
2、在target端(node3)验证分区
[root@node3 ~]# fdisk -l /dev/sdb Disk /dev/sdb: 10.7 GB, 10737418240 bytes 64 heads, 32 sectors/track, 10240 cylinders Units = cylinders of 2048 * 512 = 1048576 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x6202406e Device Boot Start End Blocks Id System /dev/sdb1 1 5121 5243888 83 Linux
之后,就可以正常使用IP SAN存储了,但在生产环境中,要注意的是,所有的initiator端可以将target端的硬盘挂载使用,所以要注意避免重复挂载。
initiator退出
[root@node1 ~]# iscsiadm -m node -d 1 -T iqn.2016-11.com.xj:s1.t1 -p 192.168.0.20:3260 -u Logging out of session [sid: 1, target: iqn.2016-11.com.xj:s1.t1, portal: 192.168.0.20,3260] Logout of [sid: 1, target: iqn.2016-11.com.xj:s1.t1, portal: 192.168.0.20,3260] successful. [root@node1 ~]# iscsiadm -m node -d 1 -T iqn.2016-11.com.xj:s1.t1 -p 192.168.0.20:3260 -o delete
删除target端的配置
[root@node3 ~]# tgtadm -L iscsi -m logicalunit -o delete -t 1 -l 2 [root@node3 ~]# tgtadm -L iscsi -m logicalunit -o delete -t 1 -l 1 [root@node3 ~]# tgtadm -L iscsi -m target -o delete -t 1
水平有限!如有错误!敬请指正!
本文出自 “linux启航” 博客,请务必保留此出处http://jiayimeng.blog.51cto.com/10604001/1876679
Cent OS 6上实现IP SAN存储