首页 > 代码库 > Flume总结(1)
Flume总结(1)
一、日志采集:从网络端口接收数据,下沉到logger
文件netcat-logger.conf:
1 # Name the components on this agent 2 #给那三个组件取个名字 3 a1.sources = r1 4 a1.sinks = k1 5 a1.channels = c1 6 7 # Describe/configure the source 8 #类型, 从网络端口接收数据,在本机启动, 所以localhost, type=spoolDir采集目录源,目录里有就采 9 a1.sources.r1.type = netcat 10 a1.sources.r1.bind = localhost 11 a1.sources.r1.port = 44444 12 13 # Describe the sink 14 a1.sinks.k1.type = logger 15 16 # Use a channel which buffers events in memory 17 #下沉的时候是一批一批的, 下沉的时候是一个个eventChannel参数解释: 18 #capacity:默认该通道中最大的可以存储的event数量 19 #trasactionCapacity:每次最大可以从source中拿到或者送到sink中的event数量 20 a1.channels.c1.type = memory 21 a1.channels.c1.capacity = 1000 22 a1.channels.c1.transactionCapacity = 100 23 24 # Bind the source and sink to the channel 25 a1.sources.r1.channels = c1 26 a1.sinks.k1.channel = c1
启动命令:
#告诉flum启动一个agent,指定配置参数, --name:agent的名字,
flume-ng agent --conf conf --conf-file conf/netcat-logger.conf --name a1 -Dflume.root.logger=INFO,console
传入数据:
[root@mini03 ~]# telnet localhost 44444 Trying ::1... telnet: connect to address ::1: Connection refused Trying 127.0.0.1... Connected to localhost. Escape character is ‘^]‘. hello world!^H^H^H^H^H^H^H^H^H^H^H^H^H^H OK tianjun2012! OK
控制台看到的数据
2017-05-08 13:41:35,766 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - org.apache.flume.sink.LoggerSink.process(LoggerSink.java:94)] Event: { headers:{} body: 68 65 6C 6C 6F 20 77 6F 72 6C 64 21 08 08 08 08 hello world!.... }
2017-05-08 13:41:40,153 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - org.apache.flume.sink.LoggerSink.process(LoggerSink.java:94)] Event: { headers:{} body: 74 69 61 6E 6A 75 6E 32 30 31 32 21 0D tianjun2012!. }
Flume总结(1)
声明:以上内容来自用户投稿及互联网公开渠道收集整理发布,本网站不拥有所有权,未作人工编辑处理,也不承担相关法律责任,若内容有误或涉及侵权可进行投诉: 投诉/举报 工作人员会在5个工作日内联系你,一经查实,本站将立刻删除涉嫌侵权内容。