Kibana is missing some logs however I can see that line of information is present in the logs on the server

Elastic Stack Logstash
1

I am unable to view a particular log in Kibana interface however that line of log is present in the log file of the application server.

I am searching wih the GUID parameter from below line, but unable to get it in the Kibana GUI
(17.0.redhat-630310 | LogTimestamp:2018-02-12 18:29:53.848|Severity:INFO|ManagedServer:10.89.249.37:16100|SourceSystemID:OCS|ChannelMedia:WEB|GUID:LTdNRBrkIBSsHFVvBkf8Ewxvn6Yyuqv9_4bb116)

If I search with other GUID i am able to find them. Please help

logstash-5.5.0
filebeat-5.5.0-linux-x86_64
elasticsearch-5.5.0

2

Can you confirm that the data is in Elasticsearch? Kibana is only searching against data indexed in Elasticsearch, not against the server files themselves. I would suggest confirming that whatever pipeline you have set up to ingest the logs from the application server into ES is working correctly.

3

Hi Bill,

It would be great if you can kindly guide me through this since I am newbie to kibana

4

If you use curl to execute this against your elasticsearch instance:

GET /_search?q=GUID:LTdNRBrkIBSsHFVvBkf8Ewxvn6Yyuqv9_4bb116

do you get back any hits? If you do not then the issue is something to do with logstash setup.

5

Do I run this on my logstash server?

curl -v 10.89.13.XX:9200 GET /_search?q=GUID:LTdNRBrkIBSsHFVvBkf8Ewxvn6Yyuqv9_4bb116

where 10.89.13.XX is my ES server

6

[fuseadmin@a0110pcsgmon02 ~]$ curl -v 10.89.13.27:9200/_search?q=GUID:LTdNRBrkIBSsHFVvBkf8Ewxvn6Yyuqv9_4bb116

  • About to connect() to 10.89.13.27 port 9200 (#0)
  • Trying 10.89.13.27...
  • Connected to 10.89.13.27 (10.89.13.27) port 9200 (#0)

GET /_search?q=GUID:LTdNRBrkIBSsHFVvBkf8Ewxvn6Yyuqv9_4bb116 HTTP/1.1
User-Agent: curl/7.29.0
Host: 10.89.13.27:9200
Accept: /

< HTTP/1.1 200 OK
< content-type: application/json; charset=UTF-8
< content-length: 149
<

  • Connection #0 to host 10.89.13.27 left intact
    {"took":16,"timed_out":false,"num_reduce_phases":2,"_shards":{"total":611,"successful":611,"failed":0},"hits":{"total":0,"max_score":null,"hits":}}[
7

So yeah looks like the data is just not in Elasticsearch. You need to look into the configuration of logstash and how the data is getting into Elasticsearch. The Logstash forum should be able to help you.

8

Hi Logstash Team, can you please check and help us on resolving this.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.