Kibana 4 / 5: how to show single events of mutltiple selected log files?

[asp]

Hi,

I am quite new to kibana 4 or 5. We are currently using the nagios logserver in production which is based on kibana 3. Now I want to get familar with kibana 4 / 5 for considering about switching to kibana 5 when it becomes stable.

First I should tell about one usecase we are using in kibana 3 (I haven't seen pure elastic's kibana 3, so I don't know how nagios specific our workflow is:

1. We are having logfiles of multiple modules of our application in elasticsearch.
2. We have multiple queries, which are like the following:

• show lines of logfile1 which contains String 1
• show lines of logfile 2 which contains String 2

3. We have filters, which are applied on top of all queries:

• show only lines of Stage Production
• show only lines where user_id is x

4. We have a panel of table style which shows the data likewise the discovery tab in kibana 4. Values are filtered for the queries and filters above.

So we do not need any aggregation here, but we need it for logfile analysis to combine multiple logs and sort it by timestmap. So we can see for example whats going on on client and server side as a stream.

How can I do it in kibana 4 or 5?
Do need to combine all queries into one or is there the possibility to set mutliple queries and to toggle them on / off by need?
How do I add manual filter in dashboards? I mean not by clicking on a bar for example in the visualisation. I would like to manually add it as filter.

Thanks in advance,
Andreas

[LeeDr (Lee Drengenberg)]

Hi Asp,

I'm doing some things with filters on 4.6 and I know you can also do the same on 5.0.

1. Are your logfile1 and logfile2 in the same Elasticsearch index? If not, you can make an index pattern that combines them. For example if I have indices like logfile1-2016.10.07 and logfile2-2016.10.07 I can make an index pattern in Kibana like logfile*-* and it would allow me to query across both indices on fields that are common to both.
2. On the Discover tab in Kibana when you select your index pattern and get a histogram and a table of documents, you can expand one of the documents and see a magnifying glass with a + in one and a - in the other. When you click these it creates a filter.

You can also click on a field in the left-hand field list and click on the magnifying glass there to add a filter.

You can pin these filters so they stay available for Visualizations and Dashboards.
You can add multiple filters and you can enable, disable, or invert each filter.

You can edit a filter to make your own custom query. For example, I changed the one in the screenshot above from path: "/var/log/syslog" to this query: "VirtualBox";

{
  "query": {
    "query_string": {
      "query": "VirtualBox",
      "analyze_wildcard": true
    }
  }
}

For the table of data, on Discover tab, when you mouse-over a field name on the left panel you see an "Add" button. When you click on that it replaces the default _source column with the new column. You can add multiple columns this way to customize your view.

Once you have your saved search added to a Dashboard, you can still expand any document and click to add a filter. And you can toggle them on/off and edit them.

Regards,
Lee