logstash reading source file and send data to elasticsearch but not displaying in kibanaa dashboard

[manjunath_mone]

Hi,

logstash beat pipe is sending data to elasticsearch, but not all logs displaying in the kibana, but if I restart the logstash beat pipe it will work for some time after few hours or next day again, I do face same issue that all logs won't display
any body can help

Thanks
manjunath

[ashishtiwari1993 (Ashish Tiwari)]

Few quick sanity check you can do -

1. Check from dev tools by hitting the query to Elasticsearch if you receiving proper data.
2. I assume you have added proper index pattern in Kibana discovery.
3. Check proper date filter you are applying on discover (Kibana).
4. Check if there any timezone mismatch data.

Let me know if you find anything in above steps.

[manjunath_mone]

Hi Ashish,

2,3,4 are correct have cross checked. I will check the first one, once I face the issue again today. The issue I am facing here is, if system is idle like whole night and next day the api logs are not visible in the Kibana while logstash pipe will be running fine. if I restart the logstash pipe, then kibana will display the data with out changing any thing.

[ashishtiwari1993 (Ashish Tiwari)]

Could you share your beat and logstash conf files?

[rugenl (Len Rugen)]

Does Kianna have a default time Rangel if the system is idle and logs nothing within this range, nothing shows. Does te restart generate log events?

[manjunath_mone]

Hi Ashish,

I didn't use any beat to push logs, instead the log files were acc
essing by creating symbolic link to the logstash machine and PFA config file

@Mr. Len,
The logs are generating continuously by all micro services, I do see logs individually in each of the micro service. Kibana dashboard has been configured to refresh every 1 min and display today's records, probably during night time there is no load , but in the morning time and currently I do see logstash pipe writing the logs to elastic server but not showing in the kibana, if logstash pipe restarts then it will work again.

it was showing below error in the kinbana browser

Error: Request timeout: "Aborted"
at search_interceptor_SearchInterceptor.handleSearchError (http://dcapimipras01:5602/d4d06bdf0d1d/bundles/plugin/data/1.0.0/data.plugin.js:1:389529)