Flume怎样安装部署

这篇文章主要介绍了Flume怎样安装部署，具有一定借鉴价值，感兴趣的朋友可以参考下，希望大家阅读完这篇文章之后大有收获，下面让小编带着大家一起了解一下。

环境：

1.flume-ng-1.6.0-cdh6.7.0.tar.gz

2.jdk 1.8

1.下载安装包并解压
wget http://archive.cloudera.com/cdh6/cdh/5/flume-ng-1.6.0-cdh6.7.0.tar.gz

tar -xvf flume-ng-1.6.0-cdh6.7.0.tar.gz

rm flume-ng-1.6.0-cdh6.7.0.tar.gz

mv apache-flume-1.6.0-cdh6.7.0-bin flume

2.配置环境变量
vi /etc/profile

export FLUME_HOME=/opt/software/flume
exportPATH=.:$FLUME_HOME/bin:$KAFKA_HOME/bin::$ZOOKEEPER_HOME/bin:$PATH

source /etc/profile

3. 配置flume-env.sh文件

cd /opt/software/flume/conf/

cp flume-env.sh.template flume-env.sh
vi flume-env.sh

export HADOOP_HOME=/opt/software/hadoop
export JAVA_HOME=/usr/java/jdk1.8.0_45

4.版本验证
flume-ng version
5.Flume部署示例
cd /opt/software/flume/bin/

vi netcat.conf

# Name the components on this agent

a1.sources = r1

a1.sinks = k1

a1.channels = c1

# Describe/configure the source

a1.sources.r1.type = netcat

a1.sources.r1.bind = localhost

a1.sources.r1.port = 44444

# Describe the sink

a1.sinks.k1.type = logger

# Use a channel that buffers events in memory

a1.channels.c1.type = memory

a1.channels.c1.capacity = 1000

a1.channels.c1.transactionCapacity = 100

# Bind the source and sink to the channel

a1.sources.r1.channels = c1

a1.sinks.k1.channel = c1

6.运行FlumeAgent，监听本机的44444端口

flume-ng agent -c conf -f netcat.conf -n a1 -Dflume.root.logger=INFO,console

[root@hadoop001 bin]# flume-ng agent -c conf -f netcat.conf -n a1 -Dflume.root.logger=INFO,console
Info: Including Hadoop libraries found via (/opt/software/hadoop-2.6.0-cdh6.7.0/bin/hadoop) for HDFS access
Info: Excluding /opt/software/hadoop-2.6.0-cdh6.7.0/share/hadoop/common/lib/slf4j-api-1.7.5.jar from classpath
Info: Excluding /opt/software/hadoop-2.6.0-cdh6.7.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar from classpath
Info: Including Hive libraries found via (/opt/software/hive-1.1.0-cdh6.7.0) for Hive access

.......

18/03/25 17:52:39 INFO instrumentation.MonitoredCounterGroup: Component type: CHANNEL, name: c1 started
18/03/25 17:52:39 INFO node.Application: Starting Sink k1
18/03/25 17:52:39 INFO node.Application: Starting Source r1
18/03/25 17:52:39 INFO source.NetcatSource: Source starting
18/03/25 17:52:40 INFO source.NetcatSource: Created serverSocket:sun.nio.ch.ServerSocketChannelImpl[/127.0.0.1:44444]

7.打开另外一个client

telnet localhost 44444

[root@hadoop001 ~]# telnet localhost 44444
Trying ::1...
telnet: connect to address ::1: Connection refused
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
111

OK18/03/25 18:11:15 WARN source.NetcatSource: Client sent event exceeding the maximum length
18/03/25 18:12:50 INFO sink.LoggerSink: Event: { headers:{} body: 31 31 31 0D 111. }

8.完毕

感谢你能够认真阅读完这篇文章，希望小编分享的"Flume怎样安装部署"这篇文章对大家有帮助，同时也希望大家多多支持，关注行业资讯频道，更多相关知识等着你来学习!