In Kibana chart I want to filter 'url' field that starts with string CANCELLED so I wrote a regex: ^CANCELLED.* but when I use filter in Discover tab then I notice that filter doesn't work properly because it also accepts urls with phrase CANCELLED inside of an url.
Is it because Kibana regex uses other character than caret for the beginning of a string?
hi @MichaelDz,
how exactly are you adding this filter. Are you adding this as a string to the query-bar? Or as a filter-pill in the filter-bar under it? Could you share a screenshot, thx?
That's how I add the filter:
The same way I add the filter in Discover tab and as you can see at the screenshot below first url is filtered correctly but not the second one:
Hi @MichaelDz,
that is actually not supported. There's an outstanding feature request here: https://github.com/elastic/kibana/issues/13943
You can do regexes in the query-bar using the Lucene query syntax, but not like that using the pills in the filter bar. For some overview, see https://www.elastic.co/guide/en/kibana/current/search.html
you could type something int he query-bar like: 'url:cancelled*'
Are you saying that it's impossible to add regex filters to a chart? Are filters in logstash only workaround?
hi @MichaelDz, you can filter your data by adding a raw lucene query-string in the query-bar instead. That supports a limited version of regex (see details here https://lucene.apache.org/core/2_9_4/queryparsersyntax.html#Wildcard%20Searches). So you should be able to filter on all documents that hat a url that starts with cancelled by using the wildcard-operator.
I understand but I don't want to write query every time I go to a chart, isn't there any permanent filter that applies to a single chart on a dashboard?
You might be able to create a saved search which includes your regex filter and then build your visualisation based on this instead of using the index pattern directly.
thx @Christian_Dahlqvist!
Other alternatives;
I also tried lucene query and filter aggregation but they have the same problem as filter. ^ symbol is not working and thus the result is incorrect.
Lucene query I tried:
url:"^CANCELLED*"@MichaelDz lucene's query-syntax doesn't fully support regex, only a subset.
the ^ caret is unnecessary, it will always start matching at the start. do url: CANCELLED*
I also tried url: CANCELLED* but it changes nothing, Kibana still filters wrong urls. Seems like only workaround is to make a new index and use regex in a sql statement.
Hi Michael,
if you want to use a regex in the query you have to surround it in forward slashes (like common in many programming languages). Also since I assume you want to add the period for matching anything.
url:/CANCELLED.*/
Also in your specific case, you are not actually needing a regex and thus you can just use a plain wildcard search - which is WAY faster - as follows:
url:CANCELLED*
The mistake in the earlier comments was, that you are NOT ALLOWED to make a space after the colon, otherwise the search will search for something different.
Also matching will still depend on how your field has been indexed. So if that above doesn't reveal the correct results, could you please check the mapping for that field (GET /your-index-pattern*/_mapping in the Console under Dev Tools in Kibana). Most likely if it doesn't work your field is of type text and not of type keyword as you would require in this case.
Cheers,
Tim
Instead of url:CANCELLED* I had to use: url.keyword:CANCELLED* and now filtering is working fine 
. Thanks everyone for the help, this solution will save a lot of space on my server.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
© 2020. All Rights Reserved - Elasticsearch
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.