Kibana filter regex 'string starts with' doesn't work

Elastic Stack Kibana
1

In Kibana chart I want to filter 'url' field that starts with string CANCELLED so I wrote a regex: ^CANCELLED.* but when I use filter in Discover tab then I notice that filter doesn't work properly because it also accepts urls with phrase CANCELLED inside of an url.

Is it because Kibana regex uses other character than caret for the beginning of a string?

2

hi @MichaelDz,

how exactly are you adding this filter. Are you adding this as a string to the query-bar? Or as a filter-pill in the filter-bar under it? Could you share a screenshot, thx?

4

That's how I add the filter:
filter

The same way I add the filter in Discover tab and as you can see at the screenshot below first url is filtered correctly but not the second one:

discover
discover.PNG812×58 4.1 KB

5

Hi @MichaelDz,

that is actually not supported. There's an outstanding feature request here: https://github.com/elastic/kibana/issues/13943

You can do regexes in the query-bar using the Lucene query syntax, but not like that using the pills in the filter bar. For some overview, see https://www.elastic.co/guide/en/kibana/current/search.html

you could type something int he query-bar like: 'url:cancelled*'

6

Are you saying that it's impossible to add regex filters to a chart? Are filters in logstash only workaround?

7

hi @MichaelDz, you can filter your data by adding a raw lucene query-string in the query-bar instead. That supports a limited version of regex (see details here https://lucene.apache.org/core/2_9_4/queryparsersyntax.html#Wildcard%20Searches). So you should be able to filter on all documents that hat a url that starts with cancelled by using the wildcard-operator.

8

I understand but I don't want to write query every time I go to a chart, isn't there any permanent filter that applies to a single chart on a dashboard?

9

You might be able to create a saved search which includes your regex filter and then build your visualisation based on this instead of using the index pattern directly.

10

thx @Christian_Dahlqvist!

Other alternatives;

11

I also tried lucene query and filter aggregation but they have the same problem as filter. ^ symbol is not working and thus the result is incorrect.

Lucene query I tried:

url:"^CANCELLED*"
12

@MichaelDz lucene's query-syntax doesn't fully support regex, only a subset.

the ^ caret is unnecessary, it will always start matching at the start. do url: CANCELLED*

13

I also tried url: CANCELLED* but it changes nothing, Kibana still filters wrong urls. Seems like only workaround is to make a new index and use regex in a sql statement.

14

Hi Michael,

if you want to use a regex in the query you have to surround it in forward slashes (like common in many programming languages). Also since I assume you want to add the period for matching anything.

url:/CANCELLED.*/

Also in your specific case, you are not actually needing a regex and thus you can just use a plain wildcard search - which is WAY faster - as follows:

url:CANCELLED*

The mistake in the earlier comments was, that you are NOT ALLOWED to make a space after the colon, otherwise the search will search for something different.

Also matching will still depend on how your field has been indexed. So if that above doesn't reveal the correct results, could you please check the mapping for that field (GET /your-index-pattern*/_mapping in the Console under Dev Tools in Kibana). Most likely if it doesn't work your field is of type text and not of type keyword as you would require in this case.

Cheers,
Tim

3 Likes
15

Instead of url:CANCELLED* I had to use: url.keyword:CANCELLED* and now filtering is working fine :slight_smile: . Thanks everyone for the help, this solution will save a lot of space on my server.

2 Likes
16

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.