首页 > 代码库 > ELK日志收集部署
ELK日志收集部署
ELK日志收集部署
1. 时间同步：
ntpdate pool.ntp.org
echo ‘*/5 * * * * ntpdate pool.ntp.org’>> /var/spool/cron/root
2. 关闭防火墙和selinux
/etc/init.d/iptables stop
chkconfig iptables off
Sed –I ‘s/SELINUX=enforcing/SELINUX=disabled/g’ /etc/selinux/config
3. 安装java
http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html
tar zxf jdk-7u45-linux-x64.tar.gz
mv jdk1.8.0_92/ /usr/local/jdk
设置jdk环境变量
vi /etc/profile
-------------------------------------------------------
JAVA_HOME=/usr/local/jdk
PATH=$PATH:$JAVA_HOME/bin
CLASSPATH=.:$JAVA_HOME/lib:$JAVA_HOME/jre/lib
export JAVA_HOME PATHCLASSPATH
-------------------------------------------------------
source /etc/profile
java -version
4. 下载redis
wget http://download.redis.io/releases/redis-3.2.3.tar.gz
tar zxf redis-3.2.3.tar.gz
cd redis-3.2.3
make
make PREFIX=/usr/local/redis install
mkdir /usr/local/redis/conf
cp redis.conf /usr/local/redis/conf/redis.conf.bak
cd /usr/local/redis/conf
cp redis.conf.bak redis.conf
添加环境变量
echo ‘PATH=$PATH:/usr/local/redis/bin/‘>>/etc/profile
source /etc/profile
启动redis：
/usr/local/redis/bin/redis-server/usr/local/redis/conf/redis.conf &
5. 下载logstash elasticsearch kibana
https://www.elastic.co/downloads
elasticsearch-5.0.0.tar.gz
logstash-5.0.0.tar.gz
kibana-5.0.0-linux-x86_64.tar.gz
6. 解压文件：
tar zxf logstash-5.0.0.tar.gz
tar zxf elasticsearch-5.0.0.tar.gz
tar zxf kibana-5.0.0-linux-x86_64.tar.gz
7. 移动到统一管理目录：
mv elasticsearch-5.0.0 /usr/local/elasticsearch
mv logstash-5.0.0 /usr/local/logstash
mv kibana-5.0.0-linux-x86_64 /usr/local/kibana
8. 备份配置文件：
cp /usr/local/logstash/config/logstash.yml /usr/local/logstash/config/logstash.yml.bak.$(date+%F)
cp /usr/local/elasticsearch/config/elasticsearch.yml/usr/local/elasticsearch/config/elasticsearch.yml.bak.$(date +%F)
cp /usr/local/kibana/config/kibana.yml/usr/local/kibana/config/kibana.yml.bak.$(date +%F)
9. 配置Elasticsearch
创建用户
默认elasticsearch不支持root用户启动，所以需要先创建一个普通用户
groupadd elastic
useradd elastic –g elastic –M
chown -R elastic.elastic /usr/local/elasticsearch/
修改配置文件：
network.host: 192.168.0.248
http.port: 9200
su elastic
/usr/local/elasticsearch/bin/elasticsearch –d
验证启动：
curl http://localhost:9200/
添加开机启动：
echo ‘/usr/local/elasticsearch/bin/elasticsearch-d’ >>/etc/rc.local
注意错误：
ERROR: bootstrap checks failed
问题：max file descriptors [65535] for elasticsearchprocess likely too low, increase to at least [65536]
解决：vi/etc/security/limits.conf
* - nofile 65536
或者
* soft nofile 65536
* hard nofile 131072
* soft nproc 2048
* hard nproc 4096
问题：max number of threads [1024] for user[elasticsearch] likely too low, increase to at least [2048]
解决：vi/etc/security/limits.d/90-nproc.conf
* soft nproc 2048
需重启生效。
问题：max virtual memory areas vm.max_map_count [65530]likely too low, increase to at least [262144]
解决：vi/etc/sysctl.conf
vm.max_map_count=655360
sysctl -p
10. 配置kibana：
修改配置文件：
vi /usr/local/kibana/config/kibana.yml
server.port: 5601
server.host: "192.168.0.248"
elasticsearch.url: http://192.168.0.248:9200
11. 配置logstash
测试logstash：
cd logstash-5.0.0
bin/logstash -e ‘input { stdin { } } output {stdout {} }‘
hello world
2013-11-21T01:22:14.405+0000 0.0.0.0 helloworld
配置logstash服务端：
vi /usr/local/logstash/config/logstash.yml
input {
redis {
host => “192.168.0.248”
port =>6379
type =>”redis-input”
data_type=>”list”
key =>”logstash:redis”
}
}
output {
elasticsearch{
hosts => ["192.168.0.248:9200"]
}
}
客户端：
vi /usr/local/logstash/config/logstash.yml
input {
file{
type=>”nginx_access”
path=>[”/usr/local/nginx/logs/access.log”]
}
}
output {
redis{
host => “192.168.0.248”
data_type=> “list”
key=> “logstash:redis”
}
}
启动客户端：
/usr/local/logstash/bin/logstash –f /usr/local/logstash/conf/logstash.conf
或yum安装：
/usr/share/logstash/bin/logstash -f/etc/logstash/conf.d/logstash.conf
12.1
13.1
客户端yum安装：
rpm --importhttps://artifacts.elastic.co/GPG-KEY-elasticsearch
cat > /etc/yum.repos.d/logstash.repo <<EOF
[logstash-5.x]
name=Elastic repository for 5.x packages
baseurl=https://artifacts.elastic.co/packages/5.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
EOF
yum clean all
yum install logstash –y
本文出自 “王家东哥” 博客，谢绝转载！
ELK日志收集部署