首页 > 代码库 > logstash 2.2以上版本,nginx 错误日志切割
logstash 2.2以上版本,nginx 错误日志切割
网上nginx错误日期切分的版本各式各样,能用的没几个,踩过很多坑,特意记录下:
if [type] == "xx_app_nginx_error_log" { grok { patterns_dir => "/etc/logstash/conf.d/patterns" match => { "message" => "%{NGINXERROR_1}" } } mutate { #避免日期报错 gsub => ["nginxErrorDateTime","/","-"] rename=>["host", "nginxErrorHost"] } #切最后一段错误信息 if [nginxErrorInfo] {
#下面有些版本的logstash使用get会报错 ruby{ code => " new_event = LogStash::Event.new(Hash[event[‘nginxErrorInfo‘].split(‘, ‘).map{|l| l.split(‘: ‘)}]) new_event.remove(‘@timestamp‘) event.append(new_event)" } }
#下面这段是用了IPIP库,不用的话可以去掉 ipip { source => "client" target => "ipip" } grok { patterns_dir => "/etc/logstash/conf.d/patterns" match => {"request"=>"%{NGINXERROR_REQUEST}"} } #删除以下字段,减少重复内容 mutate{ remove_field => [ "message","nginxErrorInfo","request"] } }
grok详细信息
NGINXERROR_1 (?<nginxErrorDateTime>\d{4}/\d{2}/\d{2}\s+\d{2}:\d{2}:\d{2})\s+\[(?<nginxErrorType>\w+)\]\s+\S+:\s+\*\d+(?<nginxErrorMsg>[^,]+), (?<nginxErrorInfo>.*)$NGINXERROR_REQUEST "%{WORD:method} %{URIPATH:urlPath}(?:\?(?<urlParams>[^ ]+))? HTTP/%{NUMBER:httpVersion}"
输出:
"@version": "1", "@timestamp": "2016-09-07T09:46:08.078Z", "path": "/to/path/nginx_error.log", "type": "xx_app_nginx_error_log", "nginxErrorDateTime": "2016-09-07 17:46:07", "nginxErrorType": "error", "nginxErrorMsg": " recv() failed (104: Connection reset by peer) while reading response header from upstream", "nginxErrorHost": "localhost", "client": "223.155.36.205", "server": "xxxxx", "upstream": "\"fastcgi://127.0.0.1:9000\"", "host": "\"localhost\"", "ipip": { "country": "中国", "province": "湖南", "city": "邵阳", "carrier": "电信" }, "method": "GET", "urlPath": "/api/test.php", "urlParams": "action=is_remind&ajax=1&cookie=0", "httpVersion": "1.1"
logstash 2.2以上版本,nginx 错误日志切割
声明:以上内容来自用户投稿及互联网公开渠道收集整理发布,本网站不拥有所有权,未作人工编辑处理,也不承担相关法律责任,若内容有误或涉及侵权可进行投诉: 投诉/举报 工作人员会在5个工作日内联系你,一经查实,本站将立刻删除涉嫌侵权内容。