首页 > 代码库 > 关于Flume中Chanel.Selector.header解释

关于Flume中Chanel.Selector.header解释

flume内置的ChannelSelector有两种,分别是Replicating和Multiplexing。

Replicating类型的ChannelSelector会针对每一个Event,拷贝到所有的Channel中,这是默认的ChannelSelector。

replicating类型的ChannelSelector例子如下

 1 a1.sources = r1 2 a1.channels = c1 c2 # 如果有100个Event,那么c1和c2中都会有这100个事件 3  4 a1.channels.c1.type = memory 5 a1.channels.c1.capacity = 1000 6 a1.channels.c1.transactionCapacity = 100 7  8  9 a1.channels.c2.type = memory10 a1.channels.c2.capacity = 100011 a1.channels.c2.transactionCapacity = 100

 

Multiplexing类型的ChannelSelector会根据Event中Header中的某个属性决定分发到哪个Channel。

multiplexing类型的ChannelSelector例子如下:

1 a1.sources = r12 3 a1.sources.source1.selector.type = multiplexing4 a1.sources.source1.selector.header = validation # 以header中的validation对应的值作为条件5 a1.sources.source1.selector.mapping.SUCCESS = c2 # 如果header中validation的值为SUCCESS,使用c2这个channel6 a1.sources.source1.selector.mapping.FAIL = c1 # 如果header中validation的值为FAIL,使用c1这个channel7 a1.sources.source1.selector.default = c1 # 默认使用c1这个channel
a1.sources.source1.selector.header = validation # 以header中的validation对应的值作为条件

同理,如下conf文件:
 1 # example.conf: A single-node Flume configuration 2  3 # Name the components on this agent 4 a1.sources = r1 5 a1.sinks = k1 6 a1.channels = c1 7  8 # Describe/configure the source 9 a1.sources.r1.type = netcat10 a1.sources.r1.bind = localhost11 a1.sources.r1.port = 4444412 13 a1.sources.r1.interceptors = i114 a1.sources.r1.interceptors.i1.type = regex_extractor15 a1.sources.r1.interceptors.i1.regex = (\\d):(\\d):(\\d)16 a1.sources.r1.interceptors.i1.serializers = s1 s2 s317 a1.sources.r1.interceptors.i1.serializers.s1.name = ip18 a1.sources.r1.interceptors.i1.serializers.s2.name = domain19 a1.sources.r1.interceptors.i1.serializers.s3.name = course20 21 a1.sources.r1.selector.type = multiplexing22 a1.sources.r1.selector.header = course23 a1.sources.r1.selector.default = c124 25 # Describe the sink26 a1.sinks.k1.type = logger27 28 # Use a channel which buffers events in memory29 a1.channels.c1.type = memory30 a1.channels.c1.capacity = 100031 a1.channels.c1.transactionCapacity = 10032 33 # Bind the source and sink to the channel34 a1.sources.r1.channels = c135 a1.sinks.k1.channel = c1

source r1中的头部有IP、Domain和cource三种信息,而r1的selector.header = course,表示selector只对IP,Domain和Cource中的Cource进行判断选择,然后再划分channel。

关于Flume中Chanel.Selector.header解释