ELK搭建

ELK搭建

一、elasticsearch

环境安装：
node1 和node2都装上
rpm --import https://packages.elastic.co/GPG-KEY-elasticsearch

#安装yum源

cat  >> /etc/yum.repos.d/elaticsearch.repo  <

#安装

yum -y install elasticsearch redis nginx java

#生成缓存

yum makecache

#测试java

[root@elk_node1 ~]# java -versionopenjdk version "1.8.0_212"OpenJDK Runtime Environment (build 1.8.0_212-b04)OpenJDK 64-Bit Server VM (build 25.212-b04, mixed mode)[root@elk_node2 ~]# java -versionopenjdk version "1.8.0_212"OpenJDK Runtime Environment (build 1.8.0_212-b04)OpenJDK 64-Bit Server VM (build 25.212-b04, mixed mode)

配置文件设置
修改/etc/elasticsearch/elasticsearch.yml配置文件
node1配置文件

[root@elk_node1 ~]# grep '^[a-Z]' /etc/elasticsearch/elasticsearch.ymlcluster.name: huanqiu               ##集群名字node.name: elk_node1                ##节点名字path.data: /data/es-data                ##储存数据路径path.logs: /var/log/elasticsearch/      ##日志路径bootstrap.memory_lock: true         ##锁定内存，不被使用到交换分区去（通常内存不足时，休眠程序内存信息会交换到交换分区）network.host: 0.0.0.0               ##所有网络  http.port: 9200                 ##端口

node2配置文件

[root@elk_node2 ~]# grep '^[a-z]' /etc/elasticsearch/elasticsearch.ymlcluster.name: huanqiunode.name: elk_node2path.data: /data/es-datapath.logs: /var/log/elasticsearch/bootstrap.memory_lock: truenetwork.host: 0.0.0.0http.port: 9200discovery.zen.ping.multicast.enabled: falsediscovery.zen.ping.unicast.hosts: ["172.16.10.76", "172.16.10.63"]

mkdir -p /data/es-data                              #创建数据存放路径chown -R elasticsearch.elasticsearch /data/es-data      #授权systemctl start elasticsearch                           #开启服务systemctl enable elasticsearch                      #设置开机自启动ss -lntup|grep 9200                                 #查看端口tcp    LISTEN     0      50       :::9200                 :::*                   users:(("java",pid=3216,fd=93))ss -lntup|grep 9300tcp    LISTEN     0      50       :::9300                 :::*                   users:(("java",pid=3216,fd=81))

测试

[root@elk_node1 ~]# curl -I 172.16.10.76:9200HTTP/1.1 200 OKContent-Type: text/plain; charset=UTF-8Content-Length: 0[root@elk_node2 ~]# curl -I 172.16.10.63:9200HTTP/1.1 200 OKContent-Type: text/plain; charset=UTF-8Content-Length: 0

web测试：


安装插件

安装head插件

[root@elk_node1 ~]# /usr/share/elasticsearch/bin/plugin install mobz/elasticsearch-head-> Installing mobz/elasticsearch-head...Trying https://github.com/mobz/elasticsearch-head/archive/master.zip ...Downloading ..............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................DONEVerifying https://github.com/mobz/elasticsearch-head/archive/master.zip checksums if available ...NOTE: Unable to verify checksum for downloaded plugin (unable to find .sha1 or .md5 file to verify)Installed head into /usr/share/elasticsearch/plugins/head

##授权

[root@elk_node1 ~]# chown -R elasticsearch.elasticsearch /usr/share/elasticsearch/plugins/

安装kopf插件

[root@elk_node1 ~]#/usr/share/elasticsearch/bin/plugin Installing lmenezes/elasticsearch-kopf-> Installing lmenezes/elasticsearch-kopf...Trying https://github.com/lmenezes/elasticsearch-kopf/archive/master.zip ...Downloading .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................DONEVerifying https://github.com/lmenezes/elasticsearch-kopf/archive/master.zip checksums if available ...NOTE: Unable to verify checksum for downloaded plugin (unable to find .sha1 or .md5 file to verify)Installed kopf into /usr/share/elasticsearch/plugins/kopf[root@elk_node1 ~]# systemctl restart elasticsearch[root@elk_node1 ~]# ll /usr/share/elasticsearch/plugins/总用量 4drwxr-xr-x 6 elasticsearch elasticsearch 4096 6月   9 12:47 headdrwxr-xr-x 8 root          root           230 6月   9 13:04 kopf

测试验证head插件


测试kopf插件


二、logstash

环境安装：

[root@elk_node1 ~]# vim /etc/yum.repos.d/logstash.repo[logstash-2.1]name=Logstash repository for 2.1.x packagesbaseurl=http://packages.elastic.co/logstash/2.1/centosgpgchech=1gpgkey=http://packages.elastic.co/GPG-KEY-elasticsearchenabled=1

yum -y install logstash

命令行输入：
标准输入，标准输出

/opt/logstash/bin/logstash -e 'input{ stdin{} } output{ stdout{} }'

OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
aaa
bbb
ccc
Settings: Default filter workers: 1
Logstash startup completed
2019-06-09T08:42:01.876Z elk_node2 aaa
2019-06-09T08:42:01.877Z elk_node2 bbb
2019-06-09T08:42:01.878Z elk_node2 ccc

标准输入，输出rubydebug格式

/opt/logstash/bin/logstash -e 'input{ stdin{} } output{ stdout{ codec => rubydebug } }'

aaaSettings: Default filter workers: 1Logstash startup completed{       "message" => "aaa",      "@version" => "1",    "@timestamp" => "2019-06-09T08:49:48.841Z",          "host" => "elk_node2"}

标准输入，在另一台主机上输出

/opt/logstash/bin/logstash -e 'input{ stdin{} } output{ elasticsearch{ hosts => ["172.16.10.76"]} }'

logstash配置文件模式

[root@elk_node1 ~]# vim /etc/logstash/conf.d/logstash.confinput { stdin{} }output { elasticsearch {hosts=> ["172.16.10.76:9200"]} }

#执行命令

/opt/logstash/bin/logstash -f /etc/logstash/conf.d/logstash.conf

例2：

[root@elk_node1 ~]# vim /data/file.confinput {   file {        path => "/var/log/messages"        type => "system"        start_position => "beginning"}}output {   elasticsearch {        hosts => ["172.16.10.76:9200"]        index => "system-%{+YYYY.MM.dd}"}}


例2：
[root@elk_node1 ~]# vim /data/file.conf
input {
file {
path => "/var/log/messages"
type => "system"
start_position => "beginning"
}
}

input {
file {
path => "/var/log/elasticsearch/huanqiu.log"
type => "es-error"
start_position => "beginning"
}
}
output {
if [type] == "system" {
elasticsearch {
hosts => ["172.16.10.76:9200"]
index => "system-%{+YYYY.MM.dd}"
}

三、kibana

环境安装：

node1和node2安装都一样

官网地址：https://www.elastic.co/cn/downloads/past-releases#kibana

[root@elk_node2 tools]# wget https://www.elastic.co/downloads/past-releases/kibana-4-3-1[root@elk_node2 tools]# lskibana-4.3.1-linux-x64.tar.gz  mongodb-linux-x86_64-3.6.12.tgz[root@elk_node2 tools]# tar xf kibana-4.3.1-linux-x64.tar.gz [root@elk_node2 tools]# mv kibana-4.3.1-linux-x64 /usr/local/[root@elk_node2 tools]# ln -s /usr/local/kibana-4.3.1-linux-x64/ /usr/local/kibana[root@elk_node2 tools]# cd /usr/local/kibana[root@elk_node2 kibana]# lsbin     installedPlugins  node          optimize      README.txt  webpackShimsconfig  LICENSE.txt       node_modules  package.json  src[root@elk_node2 kibana]# cp config/kibana.yml config/kibana.yml.bak

#修改kibana.yml配置

[root@elk_node2 kibana]# grep '^[a-z]' config/kibana.ymlserver.port: 5601server.host: "0.0.0.0"elasticsearch.url: "http://172.16.10.76:9200"           #那个节点的IP都行kibana.index: ".kibana"

##启动服务

[root@elk_node2 ~]# /usr/local/kibana/bin/kibana [root@elk_node2 ~]# screen -ls          #屏风模式There is a screen on:        5480.pts-1.elk_node2    (Detached)1 Socket in /var/run/screen/S-root.[root@elk_node2 ~]# screen -r 5480  log   [19:21:29.954] [info][status][plugin:markdown_vis] Status changed from uninitialized to green - Ready


添加索引


查看数据

添加可选项