Filter by index in ES/Kibana 5.3


(Mark Staudinger) #1

Hi Folks,

I'm currently running a production server with Kibana 4.5.0 on FreeBSD (10.3)

I'm also evaluating on a test box Elastic 5.3 / Kibana 5.3 on FreeBSD (11.0)

I'm pretty jazzed about the possibilities with the timelion query/visualizations but I'm actually getting stuck on something rather more basic. I cannot seem to select an index filter from the query bar.

Given a default index of logstash-* where I have the following indexes

logstash-default-web-2017-10-01
logstash-default-ftp-2017-10-01
logstash-default-h2-2017-10-01

When viewing the default query of "" I see results from all of logstash- within the selected time period. I do see, when examining individual items in the "Discover" tab, that the _index field is properly set to, eg, logstash-default-web-2017-10-01, etc.

If I replace "" in the query bar with, say, "logstash-default-web-", this does not restrict the results - I still see the results from all logstash-* indices. This worked quite nicely in Kibana 4.5

Has something changed here? I haven't found anything obvious in the 5.X upgrade docs or the examples in the 5.X manuals.

Thanks!
-=Mark


(Lukas Olson) #2

It could be a change in Elasticsearch. Have you tried searching for _index:logstash-default-web-*?


(Mark Staudinger) #3

Yes, I did indeed try that, and neglected to mention.

Example error output: Discover: Can only use prefix queries on keyword and text fields - not on [_index] which is of type [_index]

I should also mention that I can of course create a sub-index in settings: "logstash-default-web-*" and if I select that from the pull-down index list, then of course I can get the desired filter results.

Manually creating the index is problematic for my use case - partially as the indexes are auto-generated by content, and partially because the sub-index should be transparently shown as logstash-* to the end user.

There are also other text fields that can be matched to give the same filter results, but again it's not desirable to have those queries prepended to subsequent queries and visible in the query bar, as that would interfere with presentation and usability.

e.g., "x-pool: default AND x-service: h2"

Am I the only one using the query bar in this way? Seems unlikely, but I can't rule it out :slight_smile:

Is this a known change from Kibana 4.X? Not being able to specify the index via the query bar or by specifying the index in the /elasticsearch/logstash-string-*/_field_stats query to Elasticsearch?

Cheers,
-=Mark


(Brandon Kobel) #4

@mstaudinger are you trying to filter by index in Timelion? Timelion behaves differently than Visualize/Discover/Dashboards and you can actually use something similar to the following Timelion syntax to filter by index .es(index:logstash-default-web-*)


(Mark Staudinger) #5

Hi Brandon,

No, I haven't gotten that far - still trying to restore default behavior in my application where the index is set by a proxy so that only the relevant indices are show for a particular view. This seems to be more difficult in ES5, and I suspect it will involve customizing the POST body, based on a quick inspection of the queries coming from Kibana.

I do plan on making preformed visualizations with timelion (to address the inability to set a view scale independent of the number of data points in other visualizations, namely: time period > hours with a "request / second" scale.) So thanks for the clue about timelion - I was dimly aware of that due to examples showing how the time period etc was not maintained across the other sections of Kibana, so it makes sense that the index wouldn't carry over either. So that may be another area where I will have to proxy queries to have the intended result set without showing the filter in the query for those views that display it.

Thanks,
-=Mark


(system) #6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.