千家信息网

Flume的failover和load balance怎么使用

发表于:2024-11-30 作者:千家信息网编辑
千家信息网最后更新 2024年11月30日,本篇内容主要讲解"Flume的failover和load balance怎么使用",感兴趣的朋友不妨来看看。本文介绍的方法操作简单快捷,实用性强。下面就让小编来带大家学习"Flume的failover
千家信息网最后更新 2024年11月30日Flume的failover和load balance怎么使用

本篇内容主要讲解"Flume的failover和load balance怎么使用",感兴趣的朋友不妨来看看。本文介绍的方法操作简单快捷,实用性强。下面就让小编来带大家学习"Flume的failover和load balance怎么使用"吧!

一、Load Balancing Sink Processor

===> agent_2

source ==> agent_1 ===> agent_3 ==> Sink

===> agent_4

agent_2,agent_3,agent_4 分担agent_1的数据压力

[root@hftest0001 conf]# pwd/opt/apache-flume-1.6.0-bin/conf[root@hftest0001 conf]# vi s-exec_c-m_s-avro_sp-lb.confagent.sinkgrous.lb-1.sinks = 155-avroSink 154-avroSinkagent.sinkgrous.lb-1.processor.type = load_balance#agent.sinkgrous.lb-1.processor.backoff = trueagent.sinkgrous.lb-1.processor.selector = round_robin#agent.sinkgrous.lb-1.processor.selector.maxTimeOut = 3000agent.sinks.155-avroSink.type = avroagent.sinks.155-avroSink.hostname = ${ip-155}agent.sinks.155-avroSink.port = 55555agent.sinks.155-avroSink.channel = memoryChannelagent.sinks.154-avroSink.type = avroagent.sinks.154-avroSink.hostname = ${ip-154}agent.sinks.154-avroSink.port = 55555agent.sinks.154-avroSink.channel = memoryChannelagent.channels.memoryChannel.type = memoryagent.channels.memoryChannel.capacity = 100

二、Failover Sink Processor

[root@hftest0001 conf]# vi s-exec_c-m_s-avro_sp-fo.conf agent.sinkgrous.lb-1.sinks = 155-avroSink 154-avroSinkagent.sinkgrous.lb-1.processor.type = failoveragent.sinkgrous.lb-1.processor.priority.155-avroSink = 5agent.sinkgrous.lb-1.processor.priority.154-avroSink = 10        => 设置权重agent.sinkgrous.lb-1.processor.maxpenalty = 10000agent.sinks.155-avroSink.type = avroagent.sinks.155-avroSink.hostname = ${ip-155}agent.sinks.155-avroSink.port = 55555agent.sinks.155-avroSink.channel = memoryChannelagent.sinks.154-avroSink.type = avroagent.sinks.154-avroSink.hostname = ${ip-154}agent.sinks.154-avroSink.port = 55555agent.sinks.154-avroSink.channel = memoryChannelagent.channels.memoryChannel.type = memoryagent.channels.memoryChannel.capacity = 100

到此,相信大家对"Flume的failover和load balance怎么使用"有了更深的了解,不妨来实际操作一番吧!这里是网站,更多相关内容可以进入相关频道进行查询,关注我们,继续学习!

0