Search for specific symbols like %

Arthur_Kushman · August 17, 2017, 6:55am

We are using an ES in our project and can not find the answer or an example, how to search for ints with percent character, ex.:
we place in query match "50%" ES found the 50 and 50%, as we thought it would be a special character for ES and tried out all the analyzers like whitespace, keyword, escaping character % - nothing helped.

Also, the 2nd problem is that the highlighter doesn't enclose the whole expression into tags, ex.: 50%, but we expect - 50%

Hope somebody have the exact same experience and help, thx.

abdon · August 17, 2017, 8:21am

Not sure why it doesn't work for you, but I suspect it is an analysis issue. The below works for me, applying the whitespace analyzer. A query for just "50" returns no hits, but a query for "50%" does, with the correct highlighting: "foo 50% bar"

PUT test
{
  "mappings": {
    "doc": {
      "properties": {
        "my_field": {
          "type": "text", 
          "analyzer": "whitespace"
        }
      }
    }
  }
}

PUT test/doc/1
{
  "my_field": "foo 50% bar"
}

# Returns nothing
GET test/_search
{
  "query": {
    "match": {
      "my_field": "50"
    }
  }
}

# Returns the correct doc with correct highlighting
GET test/_search
{
  "query": {
    "match": {
      "my_field": "50%"
    }
  },
  "highlight": {
    "fields": {
      "my_field": {}
    }
  }
}

Arthur_Kushman · August 18, 2017, 8:13am

We have an existing index and try to update like this:
curl -X PUT 'http://localhost:9200/spir/material/_mapping?ignore_conflicts=true' -d
'{
"material": {
"properties": {
"text": {
"type": "text",
"analyzer": "whitespace"
},
"headline": {
"type": "text", "analyzer": "whitespace"
},
"lead": {
"type": "text", "analyzer": "whitespace"
}
}
}
}'
{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"request [/spir/material/_mapping] contains unrecognized parameter: [ignore_conflicts]"}],"type":"illegal_argument_exception","reason":"request [/spir/material/_mapping] contains unrecognized parameter: [ignore_conflicts]"},"status":400}%

Then we've tried to update without ignore_conflicts=true getting another error about conflicts:
curl -X PUT 'http://localhost:9200/spir/material/_mapping?ignore_conflicts=true' -d
'{
"material": {
"properties": {
"text": {
"type": "text",
"analyzer": "whitespace"
},
"headline": {
"type": "text", "analyzer": "whitespace"
},
"lead": {
"type": "text", "analyzer": "whitespace"
}
}
}
}'
{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"request [/spir/material/_mapping] contains unrecognized parameter: [ignore_conflicts]"}],"type":"illegal_argument_exception","reason":"request [/spir/material/_mapping] contains unrecognized parameter: [ignore_conflicts]"},"status":400}

But there are no types other then material:
curl -XGET 'localhost:9200/spir?pretty'
{
"spir" : {
"aliases" : { },
"mappings" : {
"material" : {
"properties" : {
"author_string" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"genre_id" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"headline" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"id" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"keywords" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"lang" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"lead" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"location" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"news_date" : {
"type" : "long"
},
"priority" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"products" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"rubric_id" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"service_info" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"slug" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"text" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"type" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
}
},
"settings" : {
"index" : {
"creation_date" : "1503043085847",
"number_of_shards" : "5",
"number_of_replicas" : "1",
"uuid" : "uVQ-lzwqSNOpH2-AhM8tvg",
"version" : {
"created" : "5040299"
},
"provided_name" : "spir"
}
}
}
}

abdon · August 18, 2017, 8:17am

You can't dynamically change the analyzer for an existing field on an existing index. You can only define the analyzer when you create an index (or when you add a new field to an index).

Arthur_Kushman · August 18, 2017, 12:19pm

Relatively to this article I can - https://gist.github.com/nicolashery/6317643

"To update the mappings of this existing index, you need to do it for each type (here we only have the weapons type):"

$ curl -X PUT 'http://localhost:9200/thegame/weapons/_mapping?ignore_conflicts=true' -d
'{
"weapons": {
"properties": {
"name": {
"type": "string",
"analyzer": "full_name"
},
"description": {
"type": "string"
},
"category": {
"type": "string"
}
}
}
}'

abdon · August 18, 2017, 12:45pm

That guide is for version 0.9. That is a really old version. Not sure what was possible back then, but nowadays it's definitely not possible to change the mapping of existing fields dynamically.

I'd recommend that you check out the official documentation: https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-put-mapping.html#updating-field-mappings

Arthur_Kushman · August 19, 2017, 2:02pm

Back then was better, thx.
Hate ES documentation it is like waterfall of needless words.

Arthur_Kushman · August 20, 2017, 7:10am

Found docs in ES5.5 about possible updates to fields:
https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-put-mapping.html#updating-field-mappings

It is not about type though.

abdon · August 20, 2017, 12:40pm

What you need to do if you want to change the mapping of existing documents is reindex those documents to another index with the correct mapping.

So, first create a new index which you create with the new mapping. Then use the reindex API to get all documents from the old index A into the new index B. As those documents get reindexed, they will get an updated mapping applied to them. More info about the reindex API here: https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html

system · September 17, 2017, 12:40pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
How to search special characters in elasticsearch Elasticsearch	3	2900	November 12, 2020
Range Query with % symbol Elasticsearch	3	588	January 4, 2019
How can I make this search requirement work? Elasticsearch	8	357	July 6, 2017
Regexp query and analyzer Elasticsearch	1	889	August 17, 2017
_all analyzer advice Elasticsearch	8	357	July 6, 2017

Search for specific symbols like %

Related topics