Straight vs. Curly Quotes Affecting Search

Elastic Stack Kibana
4

Thanks for the additional info. Here's what I think it happening.

The curly quotes never work as expected but that fact is being hidden. Straight double quotes have a special meaning in the Query String query, they indicate a phrase search. So any text wrapped in straight double quotes (including whitespace) will be sent as a single term. Curly quotes are not special and are not recognized by the Query String query. They are treated as regular text.

So when you search for data.status=“Active” you're literally searching for the text “Active” (including the curly double quotes). When you search for data.status="Active" you're only searching for Active (no quotes) because straight double quotes are special characters in the query language.

Here's the tricky bit: I bet the mapping for data.status uses an analyzer (or uses the default analyzer) which strips characters like “”, so after analysis you're only searching for Active which is identical to our straight double quote query.

You can test this via the _analyze API:

GET <index-name>/_analyze 
{
  "field": "data.status", 
  "text": "“Active”"
}

The response will most likely look like this:

{
  "tokens": [
    {
      "token": "active",
      "start_offset": 1,
      "end_offset": 7,
      "type": "<ALPHANUM>",
      "position": 0
    }
  ]
}

Curly double quotes don't appear to work with most of the Elasticsearch metafields because they are not analyzed, hence the quotes don't get stripped and the search term does not match.

Another way to see this is to search for a value with a space in it. Try data.status=“foo bar” vs data.status="foo bar". In the first example you're actually searching for two separate terms. data.status=“foo AND bar”. In the second example you're searching for a single phrase, foo bar.

1 Like