Logstash can't send the data to elastic search

Elastic Stack Logstash
1

Hi,

Below is my logstash configuration:

input {
beats {
port => 5044
}
}

filter {
if [type] == "abc_serverlog" {
grok {
match => { "message" => "(%{TIME:logtime}\s+)?(%{WORD:loglevel}\s+)?([%{DATA:class}]\s" }
remove_field => "residual_raw1"
}
ruby {
code => ' event.set("date", Time.now.strftime("%Y-%m-%d")) '
}
mutate {
replace => ["logtime", "%{date} %{logtime}" ]
}
date {
match => ["logtime", "HH:mm:ss,SSS", "YYYY-MM-dd HH:mm:ss,SSS" ]
remove_field => [ "date" ]
}
}
}

output {
if [fields][product] == "abc" {
elasticsearch {
hosts => ["xx.xx.xx.xx:9200"]
manage_template => false
index => "web-apache"
}
}
stdout { codec => rubydebug }
}

But logstash can't create index on elastic search and logstash was able to connect to elastic search. Moreover, logstash is running as root and elastic search is running as an user ( since elastic search doesn't allow to run as root).
Can i please know why logstash wasn't able to create index specified in the config .

2

Show an example of what you get from the stdout { codec => rubydebug } output.

3

Logstash was successfully able to connect to elastic search but it can't create and send data to index in elastic search.

[2017-10-20T13:38:15,350][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>"http://192.168.56.22:9200/"}
[2017-10-20T13:38:15,516][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//xx.xx.xx.xx:9200"]}
[2017-10-20T13:38:15,681][INFO ][logstash.pipeline        ] Starting pipeline {"id"=>"main", "pipeline.workers"=>1, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>125}
[2017-10-20T13:38:15,700][INFO ][logstash.pipeline        ] Pipeline main started
[2017-10-20T13:38:15,838][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
[2017-10-20T13:38:15,929][INFO ][org.apache.kafka.clients.consumer.ConsumerConfig] ConsumerConfig values:
4

Please post an example as stated earlier.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.