Issues with Regex in Kibana

[franciobr (Frâncio Rodrigues)]

I am having a hard time using a regex pattern inside Kibana/Elasticsearch version 6.5.4. The field I am searching for has the following mapping:

"field": {
          "type": "text",
          "analyzer": "custom_analyzer"
        },

Regex searches in this field return several hits when requested straight to elasticsearch:

GET /my_index/_search
{
    "query": {
        "regexp":{
            "field": "abc[0-9]{4}"
        }
    }
}

On the other hand, in Kibana's discover/dashboard pages all queries below return empty:

original query - field:/abc[0-9]{4}/

scaped query - field:/abc\[0\-9\]\{4\}/

desperate query - field:/.*/

Inspecting the request done by kibana to elasticsearch reveals the following query:

  "query": {
    "bool": {
      "must": [
        {
          "query_string": {
            "query": "field:/abc[0-9]{4}/",
            "analyze_wildcard": true,
            "default_field": "*"
          }
        }

I expected kibana to understand the double forward slash syntax /my_query/ and make a ´regexp query´ instead of a ´query_string´. I have tried this with both query languages: "lucene", "kuery" and with the optional "experimental query features" enabled/disabled.

Digging further I found this old issue which says that elastic only runs regex into the now deprecated _all field. If this still holds true I am not sure how regex work in kibana/elastic 6.X.

What am I missing? Any help in clarifying the conditions to use regex in Kibana would be much appreciated.

I read several discussions about the subject in this forum but still couldn't figure it out.

[franciobr (Frâncio Rodrigues)]

Hello, I still haven't been able to solve this any help would be great. I wonder if it is because my analyzer's char_filter maps bars to whitespaces. The regexp query through elasticsearch does not seem to go through the analyzer because it differentiates case and it rightfully ignores my mapping char_filter. The kibana query might not be so smart. For completion, here is the analyzer I am using for this field:

 "analysis": {
            "analyzer": {
                "pn_analyzer": {
                    "type": "custom",
                    "char_filter": ["pn_whitespaces"],
                    "tokenizer": "whitespace",
                    "filter": [
                        "asciifolding",
                        "lowercase",
                        "period"
                    ]
                }
            },
            "filter" : {
                "period" : {
                    "type" : "pattern_replace",
                    "preserve_original" : "false",
                    "pattern" : "\\.$",
                    "replacement": ""
                }
            },
            "char_filter": {
                "pn_whitespaces": {
                    "type": "mapping",
                    "mappings": [
                        "!=>\\u0020",
                        "#=>\\u0020",
                        "$=>\\u0020",
                        "%=>\\u0020",
                        "&=>\\u0020",
                        "'=>\\u0020",
                        "(=>\\u0020",
                        ")=>\\u0020",
                        "*=>\\u0020",
                        "+=>\\u0020",
                        ",=>\\u0020",
                        ":=>\\u0020",
                        ";=>\\u0020",
                        "<=>\\u0020",
                        "==>\\u0020",
                        ">=>\\u0020",
                        "?=>\\u0020",
                        "@=>\\u0020",
                        "[=>\\u0020",
                        "]=>\\u0020",
                        "^=>\\u0020",
                        "_=>\\u0020",
                        "`=>\\u0020",
                        "{=>\\u0020",
                        "|=>\\u0020",
                        "}=>\\u0020",
                        "~=>\\u0020",
                        "º=>\\u0020",
                        "¨=>\\u0020",
                        "/=>\\u0020",
                        "\\u005c=>\\u0020" # contrabarra
                            ]
                        }
                    }
                }
            }

[Joe_Fleming]

The only way I know how to do this, and it seems to work when I try it, it to create a filter and then manually modify the query DSL in the filter. The query bar at the top isn't powerful enough to perform regexp queries.

In the space under the query bar you should see a link that reads "Add a filter", and when you click on it it'll try to guide you to a quick filter. In that dialog, click on "Edit Query DSL" and then you can enter the raw JSON query. Here, you can enter exactly what you entered in your manual query:

{
  "query": {
    "regexp": {
      "field": "abc[0-9]{4}"
    }
  }
}