首页 > 代码库 > fastdfs集群
fastdfs集群
基础环境说明 6台服务器 两台tracker 4台storages
| 192.168.56.131 | tracker 服务器 | edu-dfs-tracker-1 |
| 192.168.56.132 | tracker 服务器 | edu-dfs-tracker-2 |
| 192.168.56.135 | storage服务器 | edu-dfs-storage-group1-1 |
| 192.168.56.136 | storage服务器 | edu-dfs-storage-group1-2 |
| 192.168.56.137 | storage服务器 | edu-dfs-storage-group2-1 |
| 192.168.56.138 | storage服务器 | edu-dfs-storage-group2-2 |
[root@edu-dfs-tracker-1 ~]# cat /etc/hosts 192.168.56.131 edu-dfs-tracker-1 192.168.56.132 edu-dfs-tracker-2 192.168.56.135 edu-dfs-storage-group1-1 192.168.56.136 edu-dfs-storage-group1-2 192.168.56.137 edu-dfs-storage-group2-1 192.168.56.138 edu-dfs-storage-group2-2 [root@edu-dfs-tracker-1 ~]#
2.安装依赖（所有服务器）
yum -y install perl perl-devel make gcc-c++ gcc cmake
3.安装fastdfs所需要的common库（所有的服务器）
cd /usr/local/src unzip libfastcommon-master.zip cd libfastcommon-master ./make.sh ./make.sh install
4.做软连接（所有机器）
ln -s /usr/lib64/libfastcommon.so /usr/local/lib/libfastcommon.so ln -s /usr/lib64/libfastcommon.so /usr/lib/libfastcommon.so ln -s /usr/lib64/libfdfsclient.so /usr/local/lib/libfdfsclient.so ln -s /usr/lib64/libfdfsclient.so /usr/lib/libfdfsclient.so
5.安装fastdfs（所有机器）
cd /usr/local/src tar xf fastdfs-5.05.tar.gz cd fastdfs-5.05 ./make.sh ./make.sh install
6.修改启动脚本（启动脚本是在/usr/local/bin 实际是在/usr/bin）
tracker服务器两台 sed -i ‘s#/usr/local/bin#/usr/bin#g‘ /etc/init.d/fdfs_trackerd storage服务器四台 sed -i ‘s#/usr/local/bin#/usr/bin#g‘ /etc/init.d/fdfs_storaged
7.下面来配置tracker服务器两台（tracker两台服务器全都要做）
cd /etc/fdfs cp storage.conf.sample storage.conf vi storage.conf disabled=false port=22122 base_path=/fastdfs/tracker 保存以后 mkdir -p /fastdfs/tracker /etc/init.d/fdfs_trackerd start chkconfig fdfs_trackerd on 启动以后的结果,会生成data logs 这两个文件 [root@localhost fdfs]# cd /fastdfs/tracker/ [root@localhost tracker]# ls data logs [root@localhost tracker]# 查看监听的端口22122 [root@localhost tracker]# netstat -lntup Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:22122 0.0.0.0:* LISTEN 19522/fdfs_trackerd tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 786/sshd tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 870/master tcp6 0 0 :::22 :::* LISTEN 786/sshd tcp6 0 0 ::1:25 :::* LISTEN 870/master [root@localhost tracker]# 查看启动日志： [root@localhost logs]# tail -f /fastdfs/tracker/logs/trackerd.log [2017-01-22 05:20:51] INFO - FastDFS v5.05, base_path=/fastdfs/tracker, run_by_grou
8.下面来配置storage服务器两台（storage四台服务器全都要做）（同一组的做的不一样）
cd /etc/fdfs cp storage.conf.sample storage.conf
group1 的服务器 group_name=group1 group2 的服务器 group_name=group2 其余的不变 mdkir -p /fastdfs/storage 监听日志 cd /fastdfs/storage/logs [root@localhost logs]# tail -f storaged.log mkdir data path: FD ... mkdir data path: FE ... mkdir data path: FF ... data path: /fastdfs/storage/data, mkdir sub dir done. [2017-01-22 06:07:17] INFO - file: storage_param_getter.c, line: 191, use_storage_id=0, id_type_in_filename=ip, storage_ip_changed_auto_adjust=1, store_path=0, reserved_storage_space=10.00%, use_trunk_file=0, slot_min_size=256, slot_max_size=16 MB, trunk_file_size=64 MB, trunk_create_file_advance=0, trunk_create_file_time_base=02:00, trunk_create_file_interval=86400, trunk_create_file_space_threshold=20 GB, trunk_init_check_occupying=0, trunk_init_reload_from_binlog=0, trunk_compress_binlog_min_interval=0, store_slave_file_use_link=0 [2017-01-22 06:07:17] INFO - file: storage_func.c, line: 254, tracker_client_ip: 192.168.56.137, my_server_id_str: 192.168.56.137, g_server_id_in_filename: -1992775488 [2017-01-22 06:07:19] INFO - file: tracker_client_thread.c, line: 310, successfully connect to tracker server 192.168.56.132:22122, as a tracker client, my ip is 192.168.56.137 [2017-01-22 06:07:19] INFO - file: tracker_client_thread.c, line: 310, successfully connect to tracker server 192.168.56.131:22122, as a tracker client, my ip is 192.168.56.137 [2017-01-22 06:07:19] INFO - file: tracker_client_thread.c, line: 1235, tracker server 192.168.56.132:22122, set tracker leader: 192.168.56.132:22122 [2017-01-22 06:07:19] INFO - file: storage_sync.c, line: 2698, successfully connect to storage server 192.168.56.138:23000
特别说明：一台storage只会选择一个tracker服务器，一台tracker服务器挂掉，会接着选择另外一台tracker服务器
加入开机自启，以上所有的端口都要在防火墙允许放行
echo "/etc/init.d/fdfs_storaged start " >>/etc/rc.local
9查看storage服务器的具体信心，可以用于监控，任意一台storage服务器
[root@localhost logs]# /usr/bin/fdfs_monitor /etc/fdfs/storage.conf [2017-01-22 06:32:36] DEBUG - base_path=/fastdfs/storage, connect_timeout=30, network_timeout=60, tracker_server_count=2, anti_steal_token=0, anti_steal_secret_key length=0, use_connection_pool=0, g_connection_pool_max_idle_time=3600s, use_storage_id=0, storage server id count: 0 server_count=2, server_index=0 tracker server is 192.168.56.131:22122 group count: 2 Group 1: group name = group1 disk total space = 38345 MB disk free space = 36848 MB trunk free space = 0 MB storage server count = 2 active server count = 2 storage server port = 23000 storage HTTP port = 8888 store path count = 1 subdir count per path = 256 current write server index = 0 current trunk file id = 0 Storage 1: id = 192.168.56.135 ip_addr = 192.168.56.135 (edu-dfs-storage-group1-1) ACTIVE http domain = version = 5.05 join time = 2017-01-22 05:59:38 up time = 2017-01-22 06:15:24 total storage = 38345 MB free storage = 36849 MB upload priority = 10 store_path_count = 1 subdir_count_per_path = 256 storage_port = 23000 storage_http_port = 8888 current_write_path = 0 source storage id = 192.168.56.136 if_trunk_server = 0 connection.alloc_count = 256 connection.current_count = 1 connection.max_count = 1 total_upload_count = 0 success_upload_count = 0 total_append_count = 0 success_append_count = 0 total_modify_count = 0 success_modify_count = 0 total_truncate_count = 0 success_truncate_count = 0 total_set_meta_count = 0 success_set_meta_count = 0 total_delete_count = 0 success_delete_count = 0 total_download_count = 0 success_download_count = 0 total_get_meta_count = 0 success_get_meta_count = 0 total_create_link_count = 0 success_create_link_count = 0 total_delete_link_count = 0 success_delete_link_count = 0 total_upload_bytes = 0 success_upload_bytes = 0 total_append_bytes = 0 success_append_bytes = 0 total_modify_bytes = 0 success_modify_bytes = 0 stotal_download_bytes = 0 success_download_bytes = 0 total_sync_in_bytes = 0 success_sync_in_bytes = 0 total_sync_out_bytes = 0 success_sync_out_bytes = 0 total_file_open_count = 0 success_file_open_count = 0 total_file_read_count = 0 success_file_read_count = 0 total_file_write_count = 0 success_file_write_count = 0 last_heart_beat_time = 2017-01-22 06:32:26 last_source_update = 1970-01-01 08:00:00 last_sync_update = 1970-01-01 08:00:00 last_synced_timestamp = 1970-01-01 08:00:00 Storage 2: id = 192.168.56.136 ip_addr = 192.168.56.136 (edu-dfs-storage-group1-2) ACTIVE http domain = version = 5.05 join time = 2017-01-22 06:00:33 up time = 2017-01-22 06:15:17 total storage = 38345 MB free storage = 36848 MB upload priority = 10 store_path_count = 1 subdir_count_per_path = 256 storage_port = 23000 storage_http_port = 8888 current_write_path = 0 source storage id = if_trunk_server = 0 connection.alloc_count = 256 connection.current_count = 1 connection.max_count = 1 total_upload_count = 0 success_upload_count = 0 total_append_count = 0 success_append_count = 0 total_modify_count = 0 success_modify_count = 0 total_truncate_count = 0 success_truncate_count = 0 total_set_meta_count = 0 success_set_meta_count = 0 total_delete_count = 0 success_delete_count = 0 total_download_count = 0 success_download_count = 0 total_get_meta_count = 0 success_get_meta_count = 0 total_create_link_count = 0 success_create_link_count = 0 total_delete_link_count = 0 success_delete_link_count = 0 total_upload_bytes = 0 success_upload_bytes = 0 total_append_bytes = 0 success_append_bytes = 0 total_modify_bytes = 0 success_modify_bytes = 0 stotal_download_bytes = 0 success_download_bytes = 0 total_sync_in_bytes = 0 success_sync_in_bytes = 0 total_sync_out_bytes = 0 success_sync_out_bytes = 0 total_file_open_count = 0 success_file_open_count = 0 total_file_read_count = 0 success_file_read_count = 0 total_file_write_count = 0 success_file_write_count = 0 last_heart_beat_time = 2017-01-22 06:32:18 last_source_update = 1970-01-01 08:00:00 last_sync_update = 1970-01-01 08:00:00 last_synced_timestamp = 1970-01-01 08:00:00 Group 2: group name = group2 disk total space = 38345 MB disk free space = 36849 MB trunk free space = 0 MB storage server count = 2 active server count = 2 storage server port = 23000 storage HTTP port = 8888 store path count = 1 subdir count per path = 256 current write server index = 0 current trunk file id = 0 Storage 1: id = 192.168.56.137 ip_addr = 192.168.56.137 (edu-dfs-storage-group2-1) ACTIVE http domain = version = 5.05 join time = 2017-01-22 06:07:15 up time = 2017-01-22 06:07:15 total storage = 38345 MB free storage = 36849 MB upload priority = 10 store_path_count = 1 subdir_count_per_path = 256 storage_port = 23000 storage_http_port = 8888 current_write_path = 0 source storage id = 192.168.56.138 if_trunk_server = 0 connection.alloc_count = 256 connection.current_count = 1 connection.max_count = 1 total_upload_count = 0 success_upload_count = 0 total_append_count = 0 success_append_count = 0 total_modify_count = 0 success_modify_count = 0 total_truncate_count = 0 success_truncate_count = 0 total_set_meta_count = 0 success_set_meta_count = 0 total_delete_count = 0 success_delete_count = 0 total_download_count = 0 success_download_count = 0 total_get_meta_count = 0 success_get_meta_count = 0 total_create_link_count = 0 success_create_link_count = 0 total_delete_link_count = 0 success_delete_link_count = 0 total_upload_bytes = 0 success_upload_bytes = 0 total_append_bytes = 0 success_append_bytes = 0 total_modify_bytes = 0 success_modify_bytes = 0 stotal_download_bytes = 0 success_download_bytes = 0 total_sync_in_bytes = 0 success_sync_in_bytes = 0 total_sync_out_bytes = 0 success_sync_out_bytes = 0 total_file_open_count = 0 success_file_open_count = 0 total_file_read_count = 0 success_file_read_count = 0 total_file_write_count = 0 success_file_write_count = 0 last_heart_beat_time = 2017-01-22 06:32:20 last_source_update = 1970-01-01 08:00:00 last_sync_update = 1970-01-01 08:00:00 last_synced_timestamp = 1970-01-01 08:00:00 Storage 2: id = 192.168.56.138 ip_addr = 192.168.56.138 (edu-dfs-storage-group2-2) ACTIVE http domain = version = 5.05 join time = 2017-01-22 06:06:09 up time = 2017-01-22 06:06:09 total storage = 38345 MB free storage = 36849 MB upload priority = 10 store_path_count = 1 subdir_count_per_path = 256 storage_port = 23000 storage_http_port = 8888 current_write_path = 0 source storage id = if_trunk_server = 0 connection.alloc_count = 256 connection.current_count = 1 connection.max_count = 1 total_upload_count = 0 success_upload_count = 0 total_append_count = 0 success_append_count = 0 total_modify_count = 0 success_modify_count = 0 total_truncate_count = 0 success_truncate_count = 0 total_set_meta_count = 0 success_set_meta_count = 0 total_delete_count = 0 success_delete_count = 0 total_download_count = 0 success_download_count = 0 total_get_meta_count = 0 success_get_meta_count = 0 total_create_link_count = 0 success_create_link_count = 0 total_delete_link_count = 0 success_delete_link_count = 0 total_upload_bytes = 0 success_upload_bytes = 0 total_append_bytes = 0 success_append_bytes = 0 total_modify_bytes = 0 success_modify_bytes = 0 stotal_download_bytes = 0 success_download_bytes = 0 total_sync_in_bytes = 0 success_sync_in_bytes = 0 total_sync_out_bytes = 0 success_sync_out_bytes = 0 total_file_open_count = 0 success_file_open_count = 0 total_file_read_count = 0 success_file_read_count = 0 total_file_write_count = 0 success_file_write_count = 0 last_heart_beat_time = 2017-01-22 06:32:11 last_source_update = 1970-01-01 08:00:00 last_sync_update = 1970-01-01 08:00:00 last_synced_timestamp = 1970-01-01 08:00:00
10.上传文件测试：
在tracker服务器上测试（任意一台） cd /etc/fdfs [root@localhost fdfs]# pwd /etc/fdfs [root@localhost fdfs]# ls client.conf client.conf.sample storage.conf.sample tracker.conf tracker.conf.sample [root@localhost fdfs]# cp client.conf.sample client.conf [root@localhost fdfs]#vi client.conf base_path=/fastdfs/tracker tracker_server=192.168.56.131:22122 tracker_server=192.168.56.132:22122 [root@localhost fdfs]# /usr/bin/fdfs_upload_file /etc/fdfs/client.conf /usr/local/src/fastdfs-5.05.tar.gz group2/M00/00/00/wKg4iViD5pGASNhIAAUggSnIHZU.tar.gz [root@localhost fdfs]# /usr/bin/fdfs_upload_file /etc/fdfs/client.conf /usr/local/src/libfastcommon-master.zip group1/M00/00/00/wKg4h1iD5rGATHRxAAKe64wyZIg888.zip storage服务器检测一下： [root@localhost logs]# cd /fastdfs/storage/data/00/00 [root@localhost 00]# ls wKg4h1iD5rGATHRxAAKe64wyZIg888.zip [root@localhost 00]#
11.在每个storage服务器安装nginx
fastdfs-nginx-module 安装
cd /usr/local/src/ tar xf fastdfs-nginx-module_v1.16.tar.gz cd fastdfs-nginx-module/src/ vi config CORE_INCS="$CORE_INCS /usr/include/fastdfs /usr/include/fastcommon/" #改成如下
12.nginx的安装，每个storage服务器都得安装
yum -y install openssl openssl-devel pcre* zlib libtool autoconf automke cd /usr/local/src/ tar xf nginx-1.6.2.tar.gz cd nginx-1.6.2 ./configure --add-module=/usr/local/src/fastdfs-nginx-module/src/ make && make install
13.拷贝nginx-module配置文件 storage服务器
cp /usr/local/src/fastdfs-nginx-module/src/mod_fastdfs.conf /etc/fdfs/ vi /etc/fdfs/mod_fastdfs.conf group1的配置文件 [root@localhost fdfs]# grep -r "^[a-Z]" mod_fastdfs.conf connect_timeout=2 network_timeout=30 base_path=/tmp load_fdfs_parameters_from_tracker=true storage_sync_file_max_delay = 86400 use_storage_id = false storage_ids_filename = storage_ids.conf tracker_server=192.168.56.131:22122 tracker_server=192.168.56.132:22122 storage_server_port=23000 group_name=group1 #group1 的配置文件为group1 group2的配置文件为group2 其余不变 url_have_group_name = true store_path_count=1 store_path0=/fastdfs/storage log_level=info log_filename= response_mode=proxy if_alias_prefix= flv_support = true flv_extension = flv group_count = 2 [group1] group_name=group1 storage_server_port=23000 store_path_count=1 store_path0=/fastdfs/storage [group2] group_name=group2 storage_server_port=23000 store_path_count=1 store_path0=/fastdfs/storage [root@localhost fdfs]# group2的配置文件 [root@localhost fdfs]# grep -r "^[a-Z]" mod_fastdfs.conf connect_timeout=2 network_timeout=30 base_path=/tmp load_fdfs_parameters_from_tracker=true storage_sync_file_max_delay = 86400 use_storage_id = false storage_ids_filename = storage_ids.conf tracker_server=192.168.56.131:22122 tracker_server=192.168.56.132:22122 storage_server_port=23000 group_name=group2 url_have_group_name = true store_path_count=1 store_path0=/fastdfs/storage log_level=info log_filename= response_mode=proxy if_alias_prefix= flv_support = true flv_extension = flv group_count = 2 [group1] group_name=group1 storage_server_port=23 000 store_path_count=1 store_path0=/fastdfs/storage [group2] group_name=group2 storage_server_port=23000 store_path_count=1 store_path0=/fastdfs/storage [root@localhost fdfs]# 所有的storage服务器的nginx加入开机自启 echo "/usr/local/nginx/sbin/nginx" >>/etc/rc.local
复制fastdfs的配置文件到/etc/fdfs下
[root@localhost conf]# pwd /usr/local/src/fastdfs-5.05/conf [root@localhost conf]# [root@localhost conf]# \cp http.conf /etc/fdfs/ [root@localhost conf]# \cp mime.types /etc/fdfs/
做软连接：
ln -s /fastdfs/storage/data /fastdfs/storage/data/M00
修改nginx的配置文件
[root@edu-dfs-storage-01 conf]# pwd
/usr/local/nginx/conf
vim nginx.conf 修改如下位置
user root;
listen 8888;
location ~/group([0-9])/M00 {
ngx_fastdfs_module;
}启动nginx
[root@localhost conf]# /usr/local/nginx/sbin/nginx -t ngx_http_fastdfs_set pid=19562 nginx: the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok nginx: configuration file /usr/local/nginx/conf/nginx.conf test is successful /usr/local/nginx/sbin/nginx 启动nginx echo "/usr/local/nginx/sbin/nginx" >>/etc/rc.local
测试：刚才上传文件的路径 看上面
http://192.168.56.138:8888/group2/M00/00/00/wKg4iViD5pGASNhIAAUggSnIHZU.tar.gz
http://192.168.56.135:8888/group1/M00/00/00/wKg4h1iD5rGATHRxAAKe64wyZIg888.zip
可以看出两组文件都可以下载
本文出自 “砖家博客” 博客，请务必保留此出处http://wsxxsl.blog.51cto.com/9085838/1893646
fastdfs集群
声明：以上内容来自用户投稿及互联网公开渠道收集整理发布，本网站不拥有所有权，未作人工编辑处理，也不承担相关法律责任，若内容有误或涉及侵权可进行投诉： 投诉/举报 工作人员会在5个工作日内联系你，一经查实，本站将立刻删除涉嫌侵权内容。