Querting Kibana using grok pattern

Elastic Stack Kibana
1

We have configured ELK stack over our daily logs and using Kibana UI to perform basic search/query operation on the the set of logs.

Some of our logs have a certain field in the message while others don't. Therefore we have not configured it as a separate field while configuring Logstash.

I have logs like:

[28/Jun/2016:23:59:56 +0530] 192.168.xxx.xxx [API:Profile]get_data_login: Project password success:  9xxxxxxxxx0
[28/Jun/2016:23:59:56 +0530] 192.168.xxx.xxx [API:Profile]session_end: Project logout success:  9xxxxxxxxx0 TotalTime:1.1234

In these two logs, I wish to extract TotalTime for all session_end logs. And visualize it.

How should I do it?

I can search all the logs which are listed under session_end, however I am not able to perform grok on the set of logs.

2

Use Logstash to extract the desired fields. Kibana (and ultimately Elasticsearch) can't do it.

1 Like
3

But if I make changes to Logstash the it would take care of the current logs, not the previous logs. right?

4

Correct. You'd have to reindex the old data or live with the fact that those fields are missing.