Querting Kibana using grok pattern

We have configured ELK stack over our daily logs and using Kibana UI to perform basic search/query operation on the the set of logs.

Some of our logs have a certain field in the message while others don't. Therefore we have not configured it as a separate field while configuring Logstash.

I have logs like:

[28/Jun/2016:23:59:56 +0530] 192.168.xxx.xxx [API:Profile]get_data_login: Project password success:  9xxxxxxxxx0
[28/Jun/2016:23:59:56 +0530] 192.168.xxx.xxx [API:Profile]session_end: Project logout success:  9xxxxxxxxx0 TotalTime:1.1234

In these two logs, I wish to extract TotalTime for all session_end logs. And visualize it.

How should I do it?

I can search all the logs which are listed under session_end, however I am not able to perform grok on the set of logs.

Use Logstash to extract the desired fields. Kibana (and ultimately Elasticsearch) can't do it.

1 Like

But if I make changes to Logstash the it would take care of the current logs, not the previous logs. right?

Correct. You'd have to reindex the old data or live with the fact that those fields are missing.