Logstash can't send the data to elastic search

[kenny123]

Hi,

Below is my logstash configuration:

input {
beats {
port => 5044
}
}

filter {
if [type] == "abc_serverlog" {
grok {
match => { "message" => "(%{TIME:logtime}\s+)?(%{WORD:loglevel}\s+)?([%{DATA:class}]\s" }
remove_field => "residual_raw1"
}
ruby {
code => ' event.set("date", Time.now.strftime("%Y-%m-%d")) '
}
mutate {
replace => ["logtime", "%{date} %{logtime}" ]
}
date {
match => ["logtime", "HH:mm:ss,SSS", "YYYY-MM-dd HH:mm:ss,SSS" ]
remove_field => [ "date" ]
}
}
}

output {
if [fields][product] == "abc" {
elasticsearch {
hosts => ["xx.xx.xx.xx:9200"]
manage_template => false
index => "web-apache"
}
}
stdout { codec => rubydebug }
}

But logstash can't create index on elastic search and logstash was able to connect to elastic search. Moreover, logstash is running as root and elastic search is running as an user ( since elastic search doesn't allow to run as root).
Can i please know why logstash wasn't able to create index specified in the config .

[magnusbaeck (Magnus Bäck)]

Show an example of what you get from the stdout { codec => rubydebug } output.

[kenny123]

Logstash was successfully able to connect to elastic search but it can't create and send data to index in elastic search.

[2017-10-20T13:38:15,350][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>"http://192.168.56.22:9200/"}
[2017-10-20T13:38:15,516][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//xx.xx.xx.xx:9200"]}
[2017-10-20T13:38:15,681][INFO ][logstash.pipeline        ] Starting pipeline {"id"=>"main", "pipeline.workers"=>1, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>125}
[2017-10-20T13:38:15,700][INFO ][logstash.pipeline        ] Pipeline main started
[2017-10-20T13:38:15,838][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
[2017-10-20T13:38:15,929][INFO ][org.apache.kafka.clients.consumer.ConsumerConfig] ConsumerConfig values:

[magnusbaeck (Magnus Bäck)]

Please post an example as stated earlier.