Straight vs. Curly Quotes Affecting Search

Elastic Stack Kibana
1

Hello! I have noticed that when I enter search parameters in a visualisation or in the Discovery pane in Kibana, the quotation marks being straight vs. curly affect whether or not the search is valid. However, it seems inconsistent as to which is valid; sometimes the straight quotation marks work and sometimes the curly ones work.

Why does the quotation mark style affect the search? I have had to go back to my visualisations and redo all the searches multiple times to change the quotation mark style. Is there something I can change so that both types of quotation marks are valid?

2

Could you provide some examples of queries that fail?

3

Hi Matt,

Here are some examples from our data:

data.status=“Active” and data.status="Active" are both okay.

_type:“personal” fails whereas _type:"personal" returns documents
_index:“event” vs. _index:"event" similarly

Both types of quotes seem to work for our custom fields, but curly quotes (ASCII value 8220 and 8221) fail when used with Elasticsearch's default fields, such as _type and _index.

4

Thanks for the additional info. Here's what I think it happening.

The curly quotes never work as expected but that fact is being hidden. Straight double quotes have a special meaning in the Query String query, they indicate a phrase search. So any text wrapped in straight double quotes (including whitespace) will be sent as a single term. Curly quotes are not special and are not recognized by the Query String query. They are treated as regular text.

So when you search for data.status=“Active” you're literally searching for the text “Active” (including the curly double quotes). When you search for data.status="Active" you're only searching for Active (no quotes) because straight double quotes are special characters in the query language.

Here's the tricky bit: I bet the mapping for data.status uses an analyzer (or uses the default analyzer) which strips characters like “”, so after analysis you're only searching for Active which is identical to our straight double quote query.

You can test this via the _analyze API:

GET <index-name>/_analyze 
{
  "field": "data.status", 
  "text": "“Active”"
}

The response will most likely look like this:

{
  "tokens": [
    {
      "token": "active",
      "start_offset": 1,
      "end_offset": 7,
      "type": "<ALPHANUM>",
      "position": 0
    }
  ]
}

Curly double quotes don't appear to work with most of the Elasticsearch metafields because they are not analyzed, hence the quotes don't get stripped and the search term does not match.

Another way to see this is to search for a value with a space in it. Try data.status=“foo bar” vs data.status="foo bar". In the first example you're actually searching for two separate terms. data.status=“foo AND bar”. In the second example you're searching for a single phrase, foo bar.

1 Like
5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.