Kibana Lucene query string does not match the result

[f26227279 (張皓翔)]

Kibana Lucene query string:
host:*AGC* AND NOT host:*LGAGC* AND NOT host:*AP* AND message:"\:ORA\-" AND NOT message:"ReconnectableOraErrCodes"
However, the query result not 100% match, such the pattern below in document will be returned
kQJkQJH+oRA/PDS
reference: Lucene query strings
Thank you in advance

[f26227279 (張皓翔)]

Is there anyone has some idea~?

[Christian_Dahlqvist]

Please provide full mappings for the index as well as sample documents that are or are not returned by the query incorrectly. This way someone can reproduce the issue locally and provide insight.

Although it may not necessarily matter here it also always help to state which version of Elasticsearch and Kibana you are using.

[f26227279 (張皓翔)]

doc message:
223827 controlRuleXML: <Script Type="XML" Version="2023.327.0" FilterString="">H4sIAAAAAAAAAO1daW/jOLQJkQJH+oRA/PDS6LVWIDzm3GnW/DpvCPYlu
mapping:

"message_field": {
            "path_match": "message",
            "match_mapping_type": "string",
            "mapping": {
              "norms": false,
              "type": "text"
}

Kibana & Elasticsearch Version: 7.17.2

[Christian_Dahlqvist]

The field you are searching is indexed as text, which means it is analysed using the standrard analyser. The analysis is done at both query and index time, so you can use the analyze API to find out how the queries and indexed string are tokenized.

If you analyze the message as follows:

GET /_analyze
{
  "analyzer" : "standard",
  "text" : """223827 controlRuleXML: <Script Type="XML" Version="2023.327.0" FilterString="">H4sIAAAAAAAAAO1daW/jOLQJkQJH+oRA/PDS6LVWIDzm3GnW/DpvCPYlu"""
}

you can see it generates the following token:

{
      "token" : "ora",
      "start_offset" : 108,
      "end_offset" : 111,
      "type" : "<ALPHANUM>",
      "position" : 10
},

When you run the query, the query string will also be tokenized using the standard analyzer. If you analyse the query string as follows

GET /_analyze
{
  "analyzer" : "standard",
  "text" : "\\:ORA\\-"
}

you can see that this also results in an "ora" token, which results in a match

{
      "token" : "ora",
      "start_offset" : 2,
      "end_offset" : 5,
      "type" : "<ALPHANUM>",
      "position" : 0
}

I believe this is why you get a match.