Elastic 6.2 and logstash

[rajsolanki (rajesh)]

After getting frustrated with very very simple configuration (Elastic node hangs (i think) ) i installed 6.2 version of ES , Kibana and Logstash.

ES is running in debug mode.
Logstash has debug on for following.

"logstash.outputs.elasticsearch" : "DEBUG",

I start log stash and there is no debug entries for reading my web server logs and sending event to ES.

but if i stop 6.2 version fo logstash and start 6.1 i see logs sending events to my ES.

just dont know what am i doing wrong

[rajsolanki (rajesh)]

reinstalled everything (ES , logstash and kibana). Now i m back to my origional issue which was reported in Elastic node hangs (i think). So i have no idea what am i doing wrong ( There is nothing in grok failure logs , my logstash has debug on and has lines of my http server log entries. but in ES it just doesnt show up after some time.

any body experience this ?

[rajsolanki (rajesh)]

here is hot thread info

::: {fj1omh2}{fj1omh2OR92wdgceGjnM6Q}{p8z8fkGvRDmUExLz_pydjA}{10.2.12.120}{10.2.12.120:9300}
Hot threads at 2018-02-07T21:29:39.689Z, interval=500ms, busiestThreads=3, ignoreIdleThreads=true:

0.0% (86.6micros out of 500ms) cpu usage by thread 'elasticsearch[fj1omh2][[timer]]'
 10/10 snapshots sharing following 2 elements
   java.lang.Thread.sleep(Native Method)
   org.elasticsearch.threadpool.ThreadPool$CachedTimeThread.run(ThreadPool.java:541)

this is my logstash settings output from ES

{
"logstash-2018.02.07" : {
"settings" : {
"index" : {
"refresh_interval" : "-1",
"number_of_shards" : "5",
"provided_name" : "logstash-2018.02.07",
"creation_date" : "1518025677242",
"number_of_replicas" : "1",
"uuid" : "IuhoKe4wRqWG47M1YQsV_A",
"version" : {
"created" : "6020099"
}
}
}
}
}

[warkolm (Mark Walkom)]

@rajsolanki

"refresh_interval" : "-1",

Did you set that? If so it's going to mean events (documents) will not show up even though they are being received by Elasticsearch. It needs to be a positive number.

[rajsolanki (rajesh)]

so positive number is represented in # of seconds ?

[warkolm (Mark Walkom)]

Yep.