Hi I'm student, I'm new to ELK, I would like to ask about my configurations, because the elasticsearch didn't show the indices
log from server SOA -> server ELK -> logstash -> elasticsearch
I'm using filebeat 6.4.2
the config on filebeat.yml:
#=========================== Filebeat inputs =============================
filebeat.inputs:
Each - is an input. Most options can be set at the input level, so
you can use different inputs for various configurations.
Below are the input specific configurations.
- type: log
enabled: true
paths:- /SOADEV/default/logs/server.log
fields_under_root: true
fields.task: server-soa
- /SOADEV/default/logs/server.log
#and then I want to send filebeat to logstash of ELK server
#----------------------------- Logstash output --------------------------------
output.logstash:
The Logstash hosts
hosts: ["172.16.4.20:5044"]
ssl:certificate_authorities: ["/etc/filebeat/logstash.crt"]
for logstash configuration or soatest.conf
input {
beats{
port => 5044
}
}
filter{
if [fields.task] == "server-soa"{
grok {
match => { "message" => [ "%{TIMESTAMP_ISO8601:timestamp} %{WORD:timezone} [%{DATA:isp}] %{WORD:status} | [%{DATA:externalID}] | [%{DATA:httpdo}] | [User : %{WORD:user}] | [IP : %{IP:IPclient}] | [Server : %{DATA:hostname}] | [%{DATESTAMP:timeend}] | [%{DATA:info}]", "%{TIMESTAMP_ISO8601:timestamp} %{WORD:timezone} [%{DATA:isp}] %{WORD:status} | [%{DATA:externalID}] | [%{DATA:httpdo}] | [User : %{WORD:user}] | [%{SECOND:second} Sec] | [%{DATA:info}]"
]}
}
date {
match => [ "timestamp", "yyyy-MM-dd HH:mm:ss" ]
}
}
}
output{
if [fields.task] == "server-soa" {
elasticsearch{
hosts => ["localhost:9200"]
index => "%{[fields.task]}-%{+YYYY.MM.dd}"
manage_template => false
}}
}
I made and already check the grok pattern and log, it's correct by grok debugger in dev tools
thankyou very much. I hope anyone can tell me and fix whats wrong