Kibana and old beast regexp searches

[stefws (Steffen Winther Sørensen)]

Trying to grasp why I can't do regexp searches on a field defined as:

  "message": {
    "type": "text",
    "fields": {
       "keyword": { "type" : "keyword" }
    }
  },

have read what I could find in this forum and stackoverflow, but nothing really pointed me in the right direction.

Any hints appreciated, TIA!

[flash1293 (Joe Reuter)]

Hi @stefws,

doing regexp searches on keyword fields and text fields should work fine:

PUT testindex/_doc/1
{ "test": "bacdefg" }

GET testindex/_search
{
    "query": {
        "regexp": {
            "test": {
                "value": "b.*g"
            }
        }
    }
}


GET testindex/_search
{
    "query": {
        "regexp": {
            "test.keyword": {
                "value": "b.*g"
            }
        }
    }
}

Both searches return the expected results. Do you have a specific query that's failing for you?

[stefws (Steffen Winther Sørensen)]

Just seems confusing how to ensure that Kibana Discover sends the right query using regexp...

Yes I have samples that eludes me

eg. indexes with message[.keyword] defined as above with content like:

Sep 25 09:15:27 NSX-edge-152-0 nat: SNAT_IN= OUT=vNic_0 SRC=10.10.11.10 DST=1.2.3.4 LEN=84 TOS=0x00 PREC=0x00 TTL=127 ID=31411 PROTO=UDP SPT=52493 DPT=53 LEN=64

searching in Kibana that eludes me (known CAPS is also a hazzle):

message:/Sep*SNAT*/

[flash1293 (Joe Reuter)]

Ah, maybe you didn't turn of KQL (same problem as in this topic: Kibana Regex Searching Problems)

Could you try clicking "KQL" in the top right of the search bar, set it to "Lucene" and execute the search again?

[stefws (Steffen Winther Sørensen)]

Don't see Lucene in my top right, only this in my 6.8 kibana:

[flash1293 (Joe Reuter)]

Ah, this got renamed - you have to turn off "query features" to use regular Lucene queries

[stefws (Steffen Winther Sørensen)]

same also with use of CAPS SNAT_IN=, no wanted hits though I have data as above in index:

[flash1293 (Joe Reuter)]

Could you try querying the message.keyword field instead and use .* instead of * to match unknown characters?

message.keyword:/Sep.*SNAT_IN=.*/ works for me.

[stefws (Steffen Winther Sørensen)]

Yeap that works, thanks!