類似 Splunk 的功能 OpenSource
Logstash 的使用結合 Elasticsearch 與 Kibana
Logstash + Elasticsearch + Kibana
角色說明:
(0) rsyslog [Remote linux server ]
(1) syslog collector [ Logstash ]
(2) search server [Elasticsearch ]
(3) web ui [ Kibana ]
官方實用說明
http://www.logstash.net/docs/1.4.2/tutorials/10-minute-walkthrough/
http://logstash.net/docs/1.4.2/tutorials/getting-started-with-logstash
分別使用 rpm & yum 安裝必要的軟體
(Logstash)
官網上下載該 rpm 安裝即可
( http://www.elasticsearch.org/overview/elkdownloads/ )
[root@Test-Logstash syslog]# rpm -qa | grep logstash
logstash-contrib-1.4.2-1_efd53ef.noarch
logstash-1.4.2-1_2c0f5a1.noarch
(Java)
yum 安裝即可
[root@Test-Logstash syslog]# rpm -qa | grep java
tzdata-java-2014e-1.el6.noarch
java-1.7.0-openjdk-1.7.0.65-2.5.1.2.el6_5.x86_64
(Elasticsearch)
官網上下載該 rpm 安裝即可
( http://www.elasticsearch.org/overview/elkdownloads/ )
[root@Test-Logstash src]# rpm -qa | grep elasticsearch
elasticsearch-1.3.2-1.noarch
(Kinaba)
kibana 官網上下載該檔案即可
( http://www.elasticsearch.org/overview/elkdownloads/ )
kibana-3.1.0.tar.gz 並解壓縮後放置於 /var/www/html 內即可,目錄名稱可自行命名之
(Remote Syslog Client)
修改 rsyslog.conf 並重啟該服務即可
最簡單的設定方式
在最下方加入
*.* @xxx.xxx.xxx
or
*.* @@xxx.xxx.xxx
設定 logstash 的 conf
[root@Test-Logstash conf.d]# pwd
/etc/logstash/conf.d
[root@Test-Logstash conf.d]# cat syslog.conf
input {
tcp {
type => "syslog"
port => 514
}
udp {
type => "syslog"
port => 514
}
}
filter {
if [type] == "syslog" {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
add_field => [ "received_at", "%{@timestamp}" ]
add_field => [ "received_from", "%{host}" ]
}
syslog_pri { }
date {
match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
}
output {
elasticsearch {
embedded => true
}
}
[root@Test-Logstash conf.d]#
將 logstash 服務改由 root 身份啟動
[root@Test-Logstash init.d]# pwd
/etc/init.d
[root@Test-Logstash init.d]# cat logstash
#!/bin/sh
# Init script for logstash
# Maintained by Elasticsearch
# Generated by pleaserun.
# Implemented based on LSB Core 3.1:
# * Sections: 20.2, 20.3
#
### BEGIN INIT INFO
# Provides: logstash
# Required-Start: $remote_fs $syslog
# Required-Stop: $remote_fs $syslog
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description:
# Description: Starts Logstash as a daemon.
### END INIT INFO
PATH=/sbin:/usr/sbin:/bin:/usr/bin
export PATH
if [ `id -u` -ne 0 ]; then
echo "You need root privileges to run this script"
exit 1
fi
name=root
pidfile="/var/run/$name.pid"
LS_USER=root
LS_GROUP=root
設定 Kibana config 內的 elasticsearch 的 url
[root@Test-Logstash syslog]# pwd
/var/www/html/syslog
[root@Test-Logstash syslog]# cat config.js
/** @scratch /configuration/config.js/1
*
* == Configuration
* config.js is where you will find the core Kibana configuration. This file contains parameter that
* must be set before kibana is run for the first time.
*/
define(['settings'],
function (Settings) {
/** @scratch /configuration/config.js/2
*
* === Parameters
*/
return new Settings({
/** @scratch /configuration/config.js/5
*
* ==== elasticsearch
*
* The URL to your elasticsearch server. You almost certainly don't
* want +http://localhost:9200+ here. Even if Kibana and Elasticsearch are on
* the same host. By default this will attempt to reach ES at the same host you have
* kibana installed on. You probably want to set it to the FQDN of your
* elasticsearch host
*
* Note: this can also be an object if you want to pass options to the http client. For example:
*
* +elasticsearch: {server: "http://localhost:9200", withCredentials: true}+
*
*/
elasticsearch: "http://192.168.111.145:9200",
Demo
elasticsearch test
Kibana
