首页 > 代码库 > logstash输出到influxdb

logstash输出到influxdb

2024-07-24 10:42:49   225人阅读

用了这个logstash扩展

https://github.com/PeterPaulH/logstash-influxdb/blob/master/src/influxdb.rb

把这个文件放到 logstash-1.4.2/lib/logstash/outputs

看一下午logstash的文档，终于解决了自己的需求

用python描述就是这样的

开发要求统计日志中各种类型的数量

while True:    line = f.readline()    try:        if ‘"type":"text","receiver_id"‘ in line:            type = ‘directmessage‘        elif ‘"subtype":"unfollow"‘ in line:            type = ‘unfollow‘        elif ‘"subtype":"follow"‘ in line:            type = ‘follow‘        elif ‘"subtype":"status"‘ in line:            type = ‘weibo‘        elif ‘"subtype":"comment"‘ in line:            type = ‘comment‘        else:            type = None        if type:            data = http://www.mamicode.com/["name":"pingpong_processor",                 "columns" : ["type"],                 "points" : [[type]]                }            ]

 logstash配置文件如下

input {  stdin {}}filter {  if ‘"type":"text"‘ in [message] {      mutate {        add_field => { "type" => "directmessage" }        remove_field => [ "message", "search" , "@version" ]      }  } else if ‘"subtype":"unfollow"‘ in [message] {      mutate {        add_field => { "type" => "unfollow" }        remove_field => [ "message", "search" , "@version" ]      }  } else if ‘"subtype":"follow"‘ in [message] {      mutate {        add_field => { "type" => "follow" }        remove_field => [ "message", "search" , "@version" ]      }  } else if ‘"subtype":"status"‘ in [message] {      mutate {        add_field => { "type" => "weibo" }        remove_field => [ "message", "search" , "@version" ]      }  } else if ‘"subtype":"comment"‘ in [message] {      mutate {        add_field => { "type" => "comment" }        remove_field => [ "message", "search" , "@version" ]      }  } else {      drop {}  }  }output {  influxdb {    host => "10.75.28.180"    port => 4444    name => ["pingpong_processor"]    columns => ["type", "host"]    points => [        "%{type}", "c",        "%{host}", "c"    ]  }  stdout {}}

 别忘记把influxdb的配置也修改下，因为默认upd协议是没有打开的

# Configure the udp api  [input_plugins.udp]  enabled = true  port = 4444  database = "pingpong_processor"

 我用的influxdb版本是 influxdb-0.8.2-1.x86_64,用/etc/init.d/influxdb这个启动报错，无奈手动启动的

/usr/bin/influxdb -pidfile /tmp/influxdb.pid -config config.toml

看一下udp端口是否启动了

netstat -anup|grep influxdbudp        0      0 :::4444                     :::*                                    27512/influxdb

 完工

logstash输出到influxdb


联系
我们